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1  Introduction 


The  class  of  extended  real-valued  lower  semicontinuous  (lsc)  functions  on  Mn  includes  nearly  all  func¬ 
tions  of  practical  importance  in  the  solution  of  systems  of  equations  and  inequalities,  in  optimization 
and  variational  problems,  in  statistical  estimation,  and  in  numerous  applications  related  to  data  fitting 
and  smoothing.  They  are  neither  required  to  be  integrable  nor  defined  on  a  compact  set.  The  possibility 
of  function  values  of  ±oo  facilitates  the  representation  of  constrained  minimization  problems,  where 
infeasible  points  are  assigned  the  value  oo,  and  the  utilization  of  nonlinear  transformations,  for  example 
of  the  form  g(x)  =  exp (f(x))  where  f(x)  =  — oo  is  required  to  allow  for  g(x)  =  0.  In  this  paper,  we 
develop  piecewise  polynomial  functions,  defined  on  an  arbitrary  partition  of  Mn,  that  approximate  to 
any  level  of  accuracy  a  lsc  function.  The  approximations  are  named  epi-splines  due  to  their  reliance 
on  the  epi-topology  [4,  26]  on  the  space  of  lsc  functions  and  their  obvious  structural  similarity  with 
polynomial  splines.  The  choice  of  lsc  functions  rather  than  upper  semicontinuous  (use)  functions  is 
insignificant  because  a  function  /  is  use  whenever  — /  is  lsc.  Consequently,  all  our  results  are  trivially 
extended  to  use  functions. 

We  deviate  from  the  usual  constructions  of  splines  (see,  e.g.,  [24,  10,  27,  16,  13,  20])  by  not  insisting 
on  smoothness,  a  property  not  universally  satisfied  by  the  lsc  functions.  Of  course,  the  consideration 
of  splines  with  less  than  the  standard  smoothness  is  not  new,  it  originates  with  Curry  and  Schoenberg 
[8,  9];  see  [11]  for  a  recent  study.  Our  construction  on  an  arbitrary  partition  of  the  whole  of  Mn  is 
novel,  however.  In  fact,  even  for  the  one-dimensional  special  case,  the  approximation  of  lsc  function  on 
M  is  new;  our  previous  study  [30]  dealt  with  compact  intervals  of  M. 

Splines  are  intimately  tied  to  optimization  problems  through  their  variational  theory  pioneered  in 
[18,  12,  1]  and  further  developed  up  to  the  present;  see  for  example  [21,  25,  6,  7,  5].  We  also  start 
with  a  problem  of  identifying  a  function  that  minimizes  a  criterion  and  satisfies  given  constraints, 
but  depart  from  the  traditional  focus  on  smoothness,  interpolation,  and  (least-squares)  approxima¬ 
tion.  Motivated  by  applications  in  curve  fitting,  regression,  probability  density  estimation,  variogram 
computation,  financial  curve  construction,  and  building  of  stochastic  processes  [31,  30],  we  instead 
consider  nearly  arbitrary  criterion  and  constraints,  which  leads  to  a  broad  class  of  function  identifica¬ 
tion  problems.  Although  recent  efforts  focus  on  additional  constraints,  for  example  related  to  shape 
[32,  19,  15,  14,  22,  23,  17],  our  treatment  is  more  general.  We  consider  constraints  derived  from  data 
as  well  as  extrinsic  information  about  the  actual  function  to  be  identified,  and  give  specific  examples 
related  to  continuity,  smoothness,  convexity,  monotonicity,  function  values,  unimodality,  integral  val¬ 
ues,  subgradients,  and  proximity  to  another  function.  In  contrast  to  the  common  setting  where  optimal 
solutions  of  specific  function  identification  problems  are  polynomial  splines,  the  more  general  problems, 
defined  over  the  space  of  lsc  functions  as  demanded  by  applications,  require  direct  construction  of  ap¬ 
proximating  epi-splines. 

We  establish  the  notion  of  information  growth ,  which  extends  the  traditional  study  of  limiting  properties 
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of  splines  as  additional  function  and  derivative  information  is  acquired.  We  also  consider  incrementally 
arriving  information  about  the  criterion  and  constraints  as  well  as  approximations  necessitated  by  algo¬ 
rithmic  and  computational  issues.  This  leads  to  a  rich  class  of  evolving  function  identification  problems 
and  fundamental  questions  of  approximation  of  optimization  problems.  We  establish  that  epi-splines 
obtained  as  solutions  of  such  evolving  function  identification  problems  tend  to  the  actual  function  to 
be  identified  under  general  assumptions  as  more  information  becomes  available  and  approximations  are 
refined. 

The  contribution  of  the  paper  is  therefore  three-fold:  (i)  we  construct  piecewise  polynomial  approxima¬ 
tions  of  lsc  functions,  (ii)  we  consider  general  function  identification  problems  involving  nearly  arbitrary 
constraints,  and  (iii)  we  show  that  epi-splines  obtained  by  solving  evolving  problems  of  this  general  kind 
approximate  the  actual  function  to  be  identified. 

The  paper  proceeds  in  §2  by  defining  the  broad  class  of  function  identification  problems  and  provid¬ 
ing  background  for  the  analysis.  §3  constructs  epi-splines  and  approximating  function  identification 
problems,  and  establishes  their  properties.  §4  extends  the  results  to  approximating  functions  obtained 
by  the  composition  of  a  monotonic  function  with  an  epi-spline.  §5  discusses  formulation  of  constraints 
in  function  identification  problems,  for  example  using  extrinsic  information  about  the  actual  function. 
The  paper  ends  with  numerical  examples  in  §6.  A  summary  of  some  results  and  an  exposition  of  the 
framework  of  analysis  are  given  in  the  forthcoming  tutorial  [31]. 


2  Function  Identification  Problems 

A  function  identification  problem  of  the  general  kind  considered  here  aims  to  determine  a  function  / 
on  Mn  that  minimizes  some  criterion  given  by  a  functional  -0,  while  also  satisfying  constraints  given  by 
a  set  F.  This  “actual  problem”  takes  the  form 

(. FIP )  :  min  ip(f)  such  that  /  €  F  C  F, 

where  F  is  a  space  of  functions.  We  would  like  to  handle  applications  with  discontinuous  functions 
that  might  incorporate  implicit  constraints  represented  by  the  function  value  oo.  This  situation  arises 
in  building  of  response  surfaces  defined  only  on  a  subset  set  of  Mn,  not  known  a  priori,  and  that  sub¬ 
sequently  need  to  be  minimized  under  constraints.  Nonlinear  transformations  also  demand  extended 
real- valuedness  of  functions  in  (FIP).  For  example,  a  function  /  might  be  subject  to  the  transformation 
exp (/)  to  ensure  nonnegativity  and/or  better  conditioning  but  then  /  =  — oo  is  required  to  represent 
0.  Moreover,  we  would  like  to  consider  functions  that  are  not  integrable  and  that  are  defined  on  the 
whole  of  Mn  or  unbounded  subsets.  These  requirements  lead  to  the  set  of  extended  real-valued  lower 
semi-continuous  (lsc)  functions  from  lRn  to  1R  :=  MU  {— 00,00} ,  excluding  /  =  00,  which  we  denote 
by  lsc- fens (Rn).  We  let  F  =  lsc-fcns( J?n),  or  possibly  a  subset  thereof.  The  choice  of  upper  semi- 
continuous  (use)  functions  would  yield  similar  results,  but  that  development  is  omitted  for  simplicity 
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of  exposition2. 


We  next  review  some  pertinent  facts  regarding  the  space  of  lsc  functions  and  the  notion  of  epi- 
convergence  that  is  central  in  the  subsequent  analysis;  see  [26,  Section  7.1]  for  details. 

We  embed  the  space  lsc- fens (Mn)  with  the  epi-distance  d,  which  for  any  f,g  :  lRn  -+  M,  not  identically 
equal  to  oo,  is  defined  as 

/»oo 

d{f,g)  ■■=  /  dp(f,  g)e~pdp, 

Jo 

where  the  p-epi-distance,  p  >  0,  is  given  by 

dp(f,g)  ■=  max  \d(x,  epi  /)  —  d(x,  epi  p)|, 

IMI<p 

the  standard  distance  between  a  point  x  =  (x,xq)  £  Mn+1,  with  x  £  Mn  and  xq  £  M  and  a  set 
S  C  lRn+1  is  given  by 

d(x,S)  :=  inf  \\x  -  y\\, 

y&S 

and  the  epi-graph  of  /  is  given  by 

epi /  :=  {(x,x0)  £  J?n+1  |  f(x)  <  x0}, 

with  an  analogous  expression  for  epi  <7;  see  Figure  1.  We  observe  that  dOQ(f,g)  coincides  with  the 
classical  Pompeiu-Hausdorff  distance  between  the  sets  epi  /  and  epi  g.  However,  that  notion  is  not 
appropriate  in  the  present  context  as  both  epi  /  and  epi  g  are  unbounded  and  the  “truncation”  implied 
by  a  finite  p  in  dp(f,g)  becomes  essential. 

The  epi-distance  is  a  metric  on  lsc-fcns(J?n)  and  induces  the  epi-topology  (sometimes  called  the 
Attouch-Wets  topology),  which  we  use  throughout  the  paper.  In  fact,  by  Theorem  7.58  of  [26], 
(lsc- fens (Mn),d)  is  a  complete  metric  space.  As  we  see  below,  it  is  also  separable  and  therefore  a 
Polish  space. 

The  space  lsc-fcns(J?n)  is  not  a  vector  space3  as  —  /  0  lsc- fens  (J?n)  for  /  €  lsc- fens (Kn)  given  by 
f(x)  =  0  if  x  £  [0,  oo)n  and  f{x)  =  1  otherwise.  However,  it  is  a  cone,  i.e.,  A /  £  lsc-fcns(J?n)  whenever 
A  >  0  and  /  £  lsc- fens (lRn),  and  lsc-fcns (Mn)  U  {00}  (i.e.,  the  inclusion  of  the  function  /  =  00)  is 
convex  because  A/  +  (1  —  A )g  £  lsc-fcns (lRn)  U  {00}  whenever  A  £  [0, 1]  and  f,g£  lsc-fcns (]Rn)  U  {00}. 

We  say  that  functions  fv  :  lRn  — >  M  epi-converge  to  a  function  f°  :  Mn  —tlR.  if  d(fu,f°)  — »•  0. 
By  Theorem  7.58  in  [26],  fv  epi-converges  to  f°  if  and  only  if  their  epi-graphs  converge  in  the  sense 

2Passing  to  a  “mirror”  development  for  use  functions  requires  only  adjusting  some  signs  and  inequalities,  and  replacing 
epi  by  hypo. 

3We  define  addition  of  functions  and  multiplication  with  a  scalar  in  the  usual  “pointwise”  manner.  To  handle  extended 
real  values,  we  adopt  the  conventions  that  00  +  a  =  00  and  — 00  +  a  =  —00  for  a  £  R ,  00  +  00  =  oo  +  (— 00)  =  —00  +  00  =  00, 
Aoo  =  —00  for  A  <  0,  Aoo  =  00  for  A  >  0,  Ooo  =  0,  and  similarly  for  A(— 00). 
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of  Painleve-Kuratowski4.  An  operationally  convenient  characterization  of  epi-convergence  is  resulting 
from  this  equivalence: 

2.1  Proposition  [26,  Theorem  7.2]  Functions  fv  :  Mn  — >•  1R  epiconverge  to  f  :  Hi"  — >•  Hi  if  and  only  if 
for  every  x  €  JRn, 

a)  liminfj,  fu(xu)  >  f(x),  for  all  xu  — >•  x 

b)  limsup„  fu(xu)  <  f(x)  for  some  sequence  xu  — >•  x. 

Occasionally,  we  consider  the  subset  of  lsc-fcns( J?n)  consisting  of  functions  that  take  the  value  oo  be¬ 
yond  a  closed  subset  B  of  Mn ,  which  we  denote  by  lsc-fcns(-B). 

An  actual  problem  ( FIP )  can  only  exceptionally  be  solved  analytically  and  even  computational  ap¬ 
proaches  need  to  rely  on  approximations.  Moreover,  incremental  arrival  of  data  and  extrinsic  informa¬ 
tion,  representing  information  growth  about  the  underlying  function,  might  necessitate  the  consideration 
of  a  family  of  evolving  problems.  The  formulation  and  analysis  of  such  evolving  and  approximating 
problems  are  the  topic  of  the  next  section. 

3  Evolving  Problems  and  Approximations 

In  contrast  to  the  classical  variational  theory  of  splines,  where  optimal  solutions  of  certain  problems 
automatically  result  in  piecewise  polynomial  functions  (see  for  example  [7]),  which  cannot  be  expected 
in  our  general  setting,  we  construct  piecewise  polynomial  functions  on  an  arbitrary  partition  of  Mn . 

4We  recall  that  the  outer  limit  of  a  sequence  of  sets  {A'y}vejv  is  the  collection  of  points  y  to  which  a  subsequence  of 
{y"}i^eN,  with  yv  £  AH ,  converges.  The  inner  limit  is  the  points  to  which  a  sequence  of  {y"}veN,  with  yv  £  Av ,  converges. 
If  both  limits  exist  and  are  identical,  we  say  that  the  set  is  the  Painleve-Kuratowski  limit  of  {Av}v 6jv;  see  [4,  26]. 
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The  partition  is  only  exceptionally  related  to  the  data  available  unlike  the  construction  of  classical 
univariate  splines  whose  knot  placement  is  often  data  driven.  Our  construction  leads  to  epi-splines 
and  fundamental  approximation  results  given  in  the  next  subsection.  The  subsequent  subsection  turns 
to  the  broader  question  of  evolving  criteria  and  constraints  driven  by  changes  in  approximations  and 
information  growth.  We  conclude  the  section  with  a  series  of  estimates  of  the  epi-distance  between  two 
epi-splines  as  well  as  a  discussion  of  connections  with  other  modes  of  convergence. 

3.1  Epi-splines 

Approximation  of  lsc  functions  is  achieved  by  lsc  epi-splines  that  are  piecewise  polynomial  functions 
defined  on  a  partition  of  J?n,  or  a  subset  thereof.  Specific  details  are  given  next.  We  use  the  notation 
cl  S  to  denote  the  closure  of  a  subset  S. 

3.1  Definition  (partition)  A  finite  collection  R\,R2,  ....  R.n  of  open  subsets  of  lRn  is  a  partition  of  a 
closed  set  B  C  Fin  if  U^(=1  cl  Rk  =  B  and  Rk  n  Ri  =  0  for  all  k  /  l. 

We  observe  that  the  definition  abuses  slightly  standard  terminology  by  having  the  subsets  R,\ ,  ...,  Rn 
open  and  U ^=1Rk  /  B.  However,  the  focus  on  open  sets  simplifies  the  following  exposition. 


We  adopt  a  “total  degree”  convention  and  say  that  a  polynomial  in  n  dimensions  is  of  total  degree  p  if 
it  is  expressed  as  a  finite  sum  of  polynomial  terms  each  having  the  sum  of  the  powers  of  the  variables 
being  no  larger  than  p.  Another  convention  would  have  had  only  minor  consequences  for  the  following 
results.  The  set  of  all  such  polynomials  is  denoted  by  poly p(lRn).  We  note  that  the  total  number  of 
terms  in  such  a  polynomial  is  at  most 


np  :=  (n  +  p)\ / (n\p\) . 


Consequently,  every  polynomial  in  poly p(Mn)  is  given  by  np  real  parameters.  For  any  /  :  Mn  — >  M,  we 
adopt  the  notation 

lirninf f(x')  :=  lim  inf  fix')  , 

x'^rx  54.0  x'£B(x,5) 


where 


JB{x,  5)  :=  {x'  €  Mn  |  \\x'  —  x||  <  5}. 


Informally,  lim  inf  fix')  is  the  smallest  value  of  /  near  x.  We  let  pfB  :=  ]B(0,p )  and  also  use  the 
same  notation  for  balls  in  other  dimensions  too  as  the  meaning  will  be  clear  from  the  context.  Moreover, 
IV  :=  {1,  2, ...}  and  JNq  :=  {0}  U  IV. 


3.2  Definition  (lsc  epi-splines)  A  (lsc)  epi-spline  s  :  ]Rn  —tJR.of  order  p  E  Wq,  with  partition 
1Z  =  {Rk}k=i  °f  a  closed  set  B  C  lRn,  is  a  function  that 

on  each  Rk,  k  =  1, ...,  N ,  is  polynomial  of  total  degree  p, 

has  s(x)  =  00  for  x  B,  and 

for  every  x  €  Mn ,  has  s(x)  =  lirninf  s(x'). 

x'^x 
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The  family  of  all  such  epi-splines  is  denoted  by  e-spl^  (TZ) . 

We  stress  that  B  in  Definition  3.2  might  very  well  be  the  whole  of  TR"  or  some  unbounded  subset.  As 
the  name  indicates,  lsc  epi-splines  are  indeed  lsc  functions,  which  trivially  follows  from  the  definition 
of  such  functions. 

3.3  Proposition  For  any  partition  TZ  of  a  closed  set  B  C  ]Rn ,  p  £  TNq,  and  n  €  TN,  e-spl^(7£)  C 
lsc-fcns(B)  C  lsc-fcns(Mn) . 

We  deal  exclusively  with  lsc  epi-splines  and  systematically  drop  “lsc”  in  the  following.  Since  an  s  £ 
e-spl^(7£),  with  TZ  =  {Rk}k=i>  involves  N  polynomials  of  total  degree  p,  it  is  fully  characterized  by 

ne  :=  Nnp  =  N(n  +  p)\/(n\p\)  parameters. 

We  next  show  that  epi-splines  of  any  order  can  approximate  lsc  functions  to  an  arbitrary  level  of 
accuracy.  However,  this  requires  a  refinement  of  the  partition  as  follows: 

3.4  Definition  (infinite  refinement)  A  sequence  {T Zu}<^=1  of  partitions  of  a  closed  set  B  C  Iff"  ,  with 
TZU  =  {R%} k—i,  is  an  infinite  refinement  if 

for  every  x  £  B  and  e  >  0,  there  exists  9  £  TN  such  that 
R%  C  TB(x,  e)  for  every  u  >  9  and  k  satisfying  x  £  cl  Rvk. 

In  the  case  of  a  compact  B,  there  are  obvious  choices  of  infinite  refinements.  A  simple  example  of  an 
infinite  refinement  on  (unbounded)  TR  is  to  take  Nu  =  2v  +  2,  R\  =  (— oo ,—y/v),  Rvk  =  (( k  —  v  — 
2)/y/v,  (k  —  v  —  for  k  =  2,  3, ...,  2 u  +  1,  and  R%u+2  =  °°)-  Then  9  >  max{x2,  £~2}  satisfies 

the  above  condition.  Obviously,  much  flexibility  exists  in  constructing  such  infinite  refinements.  We 
now  state  a  density  result,  which  as  elsewhere  in  the  paper  is  with  respect  to  the  epi-topology. 

3.5  Theorem  (dense  approximation)  For  any  p  £  TN o  and  an  infinite  refinement  of  a  closed 

set  B  C  TR"  , 

OO 

e-spl^'fc")  is  dense  in  lsc-fcns(B) . 

v=l 

Proof:  Let  s°  £  lsc-fcns(H)  and  TZ"  =  {Rk}k=1.  It  suffices  to  construct  a  sequence  of  epi-splines  of 
order  p  =  0.  For  every  v  £  IV  and  Rk,  k  =  1,2, ...,  Nv ,  we  define 

"infxed Rvk  s°(x)  if  infxecl^  s°(x)  £  [-v,  v\ 

°v{ R-k )  :=  (  v  if  inf^gd  Rv  s°{x)  >  v, 

—v  otherwise 

and  construct  sv  :  TR"  — »  TR  as  follows: 

su(x)  :=  ^nin  ^  ^o"(R'k)  x  £  clf?)l|,  x  £  B, 
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and  su(x)  =  oo  for  x  0  B.  Clearly,  su  is  constant  on  each  R k,  k  =  1,  2, Nv  and  satisfies  lim  irifx/^x  sv(x') 
su(x )  for  all  x  G  fRn .  Hence,  su  G  e-spl^(AT)  and  consequently  also  in  e-spl vn(fRv)  for  p  G  IN .  We  next 
show  that  the  two  conditions  of  Proposition  2.1  holds.  Let  x  €  be  arbitrary.  By  lower  semicontinu¬ 
ity  of  s°,  for  every  e  >  0  there  exists  5  >  0  such  that  s°(x')  >  s°(x)  —  s  whenever  x'  G  lB(x,6).  Since 
is  an  infinite  refinement,  there  also  exists  a  v  such  that  R k  C  JB(x,5)  for  every  v  >  D  and  k 
satisfying  x  G  cl Rk.  Hence,  there  exists  a  neighborhood  of  x  on  which  sv  >  max{s°(x)  —  e,  —  z/}  for  all 
v  >v.  Thus,  for  every  sequence  aA  — >•  x, 

limirif^  sv{xv)  >  liminf„  max{s°(x)  —  e,  — ^}  =  s°(x)  —  £. 

Since  e  is  arbitrary,  lim  inf  sv{xv)  >  s°(x)  and  condition  a)  of  Proposition  2.1  holds.  For  b),  simply 
set  xv  =  x  for  all  v.  If  x  0  B,  then  su(xu)  =  s°(x)  =  oo.  If  x  G  B  and  s°(x)  >  — oo,  then 
su(x)  <  min{s°(.x),  v}  for  all  v  sufficiently  large.  If  x  G  B  and  s°(x)  =  —  oo,  then  su(xu)  =  su(x)  =  —v. 
From  this  follows, 

lirnsup^  sv{xv)  =  limsup^  su(x)  <  s°(x), 

which  concludes  the  proof.  □ 

A  close  examination  shows  that  one  can  consider  only  rational  epi-splines  of  e-spl^(TvT)  in  the  proof 
of  Theorem  3.5,  i.e.,  functions  s  :  lRn  — >•  1R  with  s(x)  =  qu  for  x  G  R%,  qk  a  rational  constant,  k  = 
1,2,  ...,NU.  Specifically,  in  that  proof  one  can  replace  au(Rvk)  =  infoeci^  s°(x)  by  av(Rk)  equals  any 
rational  number  in  [max{— is,  inf^gciR^  s°(x)  —  1/z/},  infxgci/j^  s°(x)].  This  implies  only  minor  changes 
in  the  proof  and  we  obtain  the  next  result. 

3.6  Corollary  (separability  of  the  lsc  functions)  For  p  G  JN q  and  an  infinite  refinement 

of  a  closed  set  B  C  Mn,  (lsc-fcns(B) ,  d)  is  separable,  with  the  rational  epi-splines  of  U^T1e-spl°  (1ZU) 
furnishing  a  countable  dense  subset. 

We  observe  that  the  restrictions  to  subsets  of  lsc  functions  does  not  automatically  lead  to  similar 
density  results  as  the  following  simple  example  shows.  The  Oth-order  epi-splines  consist  of  piecewise 
constant  functions  and  the  continuous  Oth-order  epi-splines  are  therefore  simply  the  constant  functions. 
Consequently,  the  continuous  Oth-order  epi-splines  cannot  be  dense  in  the  space  of  continuous  functions. 
Often,  the  choice  of  partition  can  also  lead  to  the  situation  that  every  continuous  epi-spline,  regardless 
of  order,  is  simply  a  constant  function.  An  example  is  the  partition  of  ]R2  into  R\  =  {x  \  eXl  >  X2} 
and  R.2  =  [x  \  exi  <  x-^}-  Two  polynomials  defined  on  R\  and  R2,  respectively,  cannot  coincide  on 
{x  |  exi  =  X2}  without  being  identical  everywhere.  The  situation  further  complicates  with  restrictions 
to  continuity  of  derivatives.  We  here  give  a  result  for  a  partition  consisting  of  simplexes.  We  recall 
that  a  simplex  S  in  Mn  is  the  convex  hull  of  n  +  1  points  x°,  x1 , ...,  xn  G  IR." ,  with  x1  —  x°,  x2  —  x°,  ..., 
xn  —  x°  linearly  independent.  We  start  with  the  case  when  only  a  compact  subset  of  fRn  needs  to  be 
partitioned. 

3.7  Definition  (simplicial  complex  partition  of  compact  set.)  A  partition  R±,  R2, ...,  Rn  of  lRn  is  a 
simplicial  complex  partition  of  a  compact  set  B  C  Mn  if  cl  R\,  ...,  c\Rn  are  simplexes. 
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As  usual,  we  denote  by  C°(B)  the  continuous  functions  on  B  C  Mn .  This  leads  us  to  the  following 
density  result  for  continuous  functions,  which  holds  for  orders  p  >  1. 

3.8  Theorem  (dense  approximation  of  continuous  functions  on  compact  set)  For  any  p  €  IV  and 
{Kt'}%1,  an  infinite  refinement  of  a  compact  set  B  C  Mn  consisting  of  simplicial  complex  partitions  of 

B, 

OO 

|^J  e-spl n  C°(B)  is  dense  in  C°(B). 

u=\ 

Proof:  Let  s°  €  C°(B )  and  TZU  =  { R If  suffices  to  construct  a  sequence  of  epi-splines  of  order 
p  =  1.  For  k  =  1, ...,  Nv ,  let  q Y  be  the  unique  affine  function  on  lRn  that  coincides  with  the  values  of  s° 
at  the  n+l  extreme  points  of  R V.  We  define  sv  to  be  the  epi-spline  with  partition  TZV  given  by  the  affine 
functions  qk  on  Rlk,  k  =  1  which  is  then  of  first  order.  We  next  consider  the  continuity  of  su . 

Consider  a  facet5  {x  €  Mn  \  x  €  cl  Rk fl  cl  R" } ,  k  /  l,  which  is  necessarily  bounded.  By  construction,  qk 
and  qY  both  coincide  with  the  value  of  s°  at  the  n  extreme  points  of  the  facet.  Since  q’f  and  qY  are  affine, 
they  must  coincide  on  the  entire  facet.  Consequently,  sv  is  continuous  and  in  e-spl for  p  €  IV. 
We  next  show  that  the  two  conditions  of  Proposition  2.1  holds.  Let  x  €  B  be  arbitrary.  By  continuity 
of  s°,  for  every  e  >  0  there  exists  S  >  0  such  that  |s°(x')  —  s°(.r)|  <  e  whenever  x'  €  lB(x,  6)  fl  B.  Since 
{A+}^Li  is  an  infinite  refinement,  there  also  exists  a  v  such  that  Rvk  C  lB(x,5 )  for  every  v  >  v  and 
k  satisfying  x  €  cl  Rvk.  We  find  that  mmx>eBrx  ^nB  s°(x')  <  qk  <  m&xxieB(x,5)rB  s°(x')  for  all  every 
v  >  v  and  k  satisfying  x  €  c\Rk.  Hence,  there  exists  a  neighborhood  S  of  x  where  \su(x')  —  s°(x)|  <  £ 
for  all  v  >  v  and  x'  €  S  H  B.  This  and  the  fact  that  both  sv  and  s°  are  infinity  beyond  B ,  we  find  that 
for  every  sequence  xv  — >•  x, 

lirninf^  su(xu)  >  s°(x)  —  e. 

Since  e  is  arbitrary,  liminf  su(xu)  >  s°(x)  and  condition  a)  of  Proposition  2.1  holds.  For  b),  simply  set 
xv  =  x  for  all  v  and  observe  that  sl/(.x)  <  s°(.t)  +  e  for  all  v  >  v.  From  this  follows, 

limsupj,  su(xu)  =  lirnsup^  sv{x)  <  s°(x )  +  e. 

Again,  since  e  is  arbitrary  condition  b)  follows,  which  concludes  the  proof.  □ 

The  case  of  partition  of  the  whole  of  lRn  requires  additional  notation.  For  x  =  (aq,  ...,xn)  €  JRn ,  we 
use  the  notation  to  denote  the  (n  —  l)-dimensional  vector  that  excludes  the  ith  component  of  x. 

3.9  Definition  (simplex  cylinder.)  A  simplex  cylinder  S  in  Mn  generated  by  a  simplex  S'  in  J?”-1 
and  the  interval  [771,7/2]  C  1R  is  the  set  {x  €  Mn  \  X-i  €  S',Xi  €  [771,7/2]}  for  some  i.  We  say  that  S'  is 
the  base  of  S. 

3.10  Definition  (simplicial  complex  partition  of  fRn .)  A  partition  Ri,  R2, ...,  Rjy  of  ]Rn  is  a  simplicial 
complex  partition  of  lRn  if  for  a  box  B  =  [l\,ui]  x  \f,2,u^  x  ...  x  [ln,un\,  with  —00  <  U  <  Ui  <  00, 
7  =  1,2, ... ,  77.,  and  Ns ,  Nc,  Nr  €  IV,  with  Ns  +  Nc  +  Nr  =  N,  we  have  that 

5We  recall  that  a  facet  of  an  n-dimensional  simplex  is  an  (n  —  l)-dimensional  set  defined  by  one  of  the  faces  of  the 
simplex. 


a)  clRi,  cl  Rns  are  simplexes  and  cl  Rk  =  B 

b)  Rns+ i,  ■■■,  Rns+nc  are  simplex  cylinders  generated  by  the  (n  —  1) -dimensional  simplexes  formed  by 
the  intersection  of  Rk,  for  some  k  =  1,  ...Ns,  and  a  facet  of  B  {x  £  Mn  \  lj  <  Xj  <  Uj,j  j -  i,Xi  =  U}, 
for  some  i  =  1, n,  and  the  interval  [— oo,  /,;]  as  well  as  by  the  simplexes  formed  by  the  intersection 
of  Rk,  for  some  k  =  1,  ...Ns,  and  {x  £  Mn  \  lj  <  Xj  <  Uj,j  j -  i,Xi  =  Ui}  for  some  i  =  1,  ...,n,  and 
the  interval  [iij,oo]. 

c)  Rns+nc+i,  ■■■,  Rn  are  unbounded  n-dimensional  hyperrectangles  given  by  the  n-fold  cartesian  prod¬ 
uct  of  intervals  of  the  forms  [k,Ui\,  [u^oo),  and  (— oo  ,U],  i  =  1 ,  ...,w,  each  using  at  most  n  —  2 
intervals  of  the  form  [li,Ui] . 

A  density  result  for  continuous  functions  on  ]Rn  follows  next. 

3.11  Theorem  (dense  approximation  of  continuous  functions  on  Mn)  For  any  p  £  IV  and 
an  infinite  refinement  on  Mn  consisting  of  simplicial  complex  partitions  of  Mn, 

OO 

\J  e-spl£(7r)  nC°(JT)  is  dense  in  C°{fRn). 

V=1 

Proof:  Let  s°  £  C®(fRn)  and  1ZU  =  {RVk}k=ii  with  N”  and  N”  the  number  of  simplexes  and  simplex 
cylinders,  respectively.  Again,  it  suffices  to  construct  a  sequence  of  epi-splines  of  order  p  =  1.  For 
k  =  1  ,...,Ng,  let  q y.  be  the  unique  affine  function  on  Mn  that  coincides  with  the  values  of  s°  at  the 
n  +  1  extreme  points  of  Rvk.  For  k  =  N"  +  1, ...,  N"  +  N",  let  qk  be  the  unique  affine  function  on 
]Rn  that  coincides  with  the  values  of  s°  at  the  n  extreme  points  of  Rk  and  that  is  constant  in  the 
directions  perpendicular  to  the  base  of  the  simplex  cylinder  Rk.  For  k  =  N "  +  N”  +  1,  ...,1V,  Rvk  are 
hyperrectangles  with  n,  n  —  1,  ...3,  or  2  unbounded  directions.  If  the  number  of  unbounded  directions 
of  such  Rk  is  m,  then  Rk  has  n  —  m+l  extreme  points.  Let  qk  be  the  unique  affine  function  on  lRn  with 
value  of  s°  at  the  extreme  points  of  Rk  and  that  is  constant  in  the  directions  of  unboundedness.  We 
define  sv  to  be  the  epi-spline  given  by  the  affine  functions  q'f,  k  =  1, ...,  N,  which  is  then  of  first  order. 
We  next  consider  the  continuity  of  su .  First  consider  a  facet  {x  £  ]Rn  \  x  £  cl  Rk  n  cl  R"},  k  /  Z,  that  is 
bounded.  By  construction,  qk  and  q”  both  coincide  with  the  value  of  s°  on  the  n  extreme  points  of  the 
facet.  Since  qk  and  q”  are  affine,  the  must  coincide  on  the  whole  facet.  Second  consider  an  unbounded 
facet  {x  £  lRn  \  x  £  cl  R'k  fl  cl  R'{},  k  /  l.  By  construction,  qk  and  q”  both  coincide  with  the  value  of  s° 
on  the  extreme  points  of  the  facet.  This  fact  and  the  constancy  of  qk  and  qY  in  unbounded  directions 
imply  that  qk  and  q”  coincide  on  the  whole  facet.  Consequently,  sv  is  continuous  and  in  e-spl for 
p  £  IV.  We  next  show  that  the  two  conditions  of  Proposition  2.1  holds.  Let  x  £  Mn  be  arbitrary.  By 
continuity  of  s°,  for  every  e  >  0  there  exists  S  >  0  such  that  |s°(x')  —  s°(x)|  <  e  whenever  x'  £  lB(x,  5). 
Since  is  an  infinite  refinement,  there  also  exists  a  v  such  that  Rvk  C  JB(x,6)  for  every  u  >  v 

and  k  satisfying  x  £  cll?£.  Thus  min x'^b(x,8)  s°(x')  <  qk  <  ma xxreB(x,5)  s°(x')  for  all  every  v  >  v  and 
k  satisfying  x  £  cl-R^  and  there  exists  a  neighborhood  S'  of  a:  where  |sJ/(.'c/)  —  s°(x)|  <  e  for  all  v  >  u 
and  x'  €  S.  Consequently,  for  every  sequence  xu  — >•  x, 

liminfy  su(xu)  >  s°(x)  —  £. 
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Since  e  is  arbitrary,  liminf  sv(xv )  >  s°(x)  and  condition  a)  of  Proposition  2.1  holds.  For  b),  simply  set 
xv  =  x  for  all  v  and  observe  that  su(x)  <  s°(x)  +  £  for  all  u  >i).  From  this  follows, 

limsup„  sv(xv)  =  limsup;y  su(x)  <  s°(x)  +  e. 

Again,  since  e  is  arbitrary  condition  b)  follows,  which  concludes  the  proof.  □ 

We  end  the  subsection  with  a  discussion  of  “decomposition”  of  epi-splines  on  Mn  into  sums  of  those 
defined  on  a  lower- dimensional  space.  For  given  p,  n  €  iV  and  x  €  Mn,  there  are  (”j  subvectors  of  x 
with  p  components.  We  denote  by  j  =  1,...,  (”),  these  subvectors.  The  next  theorem  provides  a 
decomposition  of  an  epi-spline  in  terms  of  lower- dimensional  polynomials. 

3.12  Theorem  (decomposition.)  For  every  s  €  e-spl^(7£),  with  n  >  p  and  1Z  =  {Rk}^=1,  and  k  = 
1,  2, ...,  N,  there  exist  €  poly p(Mn)  and  q^.j  €  poly P(]RP),  j  =  1,  2, ...,  (”)  ,  such  that 

(;) 

s(x)  =  qk{x)  =  ^2  Qk,j(x\j]),  for  all  x  €  Rk- 
i=i 

Proof:  The  first  equality  follows  trivially  from  the  definition  of  epi-splines.  A  polynomial  qk(x)  is  the 
sum  of  np  terms  each  involving  at  most  p  components  of  x.  A  term  involving  Xj1 , ...,  Xj  ,  say,  is  also  part 
of  the  description  of  a  polynomial  of  total  degree  p  in  the  p-dimensional  subspace  of  ]Rn  corresponding 
to  dimensions  j\ ,  ...,  jp.  Since  there  are  (™)  ways  of  selecting  such  subspaces,  the  second  equality 
follows.  □ 

Theorem  3.12  decomposes  n-dimensional  polynomials  into  sums  of  p-dimensional  polynomials.  Using 
similar  arguments  as  in  this  theorem’s  proof,  we  obtain  a  “one-dimensional  reduction”  as  stated  next. 

3.13  Corollary  For  every  s  €  e-spl^(T^),  with  n  >  p  and  1Z  =  {Rk}k=i>  an(^  k  =  1,2, ...,  N,  there  exist 
qk,i  €  polyp(l?"“1),  i  =  1,2,  such  that 

n 

s(x)  =  22  <lk,i(x-i)i  for  all  x  G  Rk- 

i= 1 

3.2  Evolving  and  Approximating  Problems 

The  previous  subsection  deals  with  the  approximation  of  lsc-fcns(J?n)  by  e-spl^(7£).  This  lays  the 
foundation  for  approximating  the  infinite-dimensional  ( FIP )  by  one  involving  a  finite  number  of  pa¬ 
rameters.  In  practice,  however,  data  and  extrinsic  information  might  accumulate  gradually  and  the 
criterion  functional  and  constraints  might  also  need  to  be  approximated,  which  lead  to  a  family  of 
evolving  function  identification  problems: 

( FIPU )  :  mun/A(s)  such  that  s  €  Fu  n  Su , 
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with  ifu  :  F  — >•  M  being  an  approximate  criterion  functional,  possibly  representing  an  incomplete  data 
set  of  size  v,  Fv  C  J  being  an  approximate  constraint  set  that  substitutes  F  using  currently  avail¬ 
able  information  as  well  as  computationally  required  approximations,  and  Su  C  e-splf,  (Ttl)  fl  F .  Since 
epi-splines  are  characterized  by  a  finite  number  of  parameters,  ( FIPU )  is  in  one-to-one  correspondence 
with  an  optimization  problem  in  a  (finite-dimensional)  Euclidean  space.  If  the  approximations  are 
constructed  properly,  one  might  hope  that  solutions  of  ( FIPU )  approximate  solutions  of  ( FIP ).  This 
property  is  ensured  by  the  next  results  under  standard  conditions.  First,  however,  we  need  to  recall  an 
extended  notion  of  epi-convergence. 

Since  we  not  only  need  to  deal  with  epi-convergence  of  functions  in  lsc-fcns(-R"),  but  also  of  the 
extended  real-valued  functionals  defined  on  lsc-fcns(K"),  giving  rise  to  (FIP)  and  (FIPV),  we  provide 
a  definition  of  epi-convergence  of  the  evolving  problems  ( FIPU )  to  the  limiting  problem  (FIP);  see  [2] 
for  further  details  about  epi-convergence  of  functionals  defined  on  metric  spaces. 

3.14  Definition  A  sequence  of  evolving  problems  {(FIPu)}p£jy  epi-converges  to  an  actual  problem 
(FIP)  if 

a)  for  every  sequence  {sv}v^k,  with  K  an  infinite  subset  of  IN,  sv  G  Fu  n  Su ,  and  dL(sv ,  /)  — >•  0,  we 
have  that  f  G  F  and  liminf vifv(sv)  >  if(f); 

b)  for  every  f  G  F,  there  exists  a  sequence  with  su  £  Fu  n  Sv ,  such  that  dt(sv ,  /)  — >•  0  and 

limsup„'0i'(s1')  <  if(f). 

With  the  perspective  that  (FIPU)  is  an  approximation  of  (FIP),  we  give  two  results  that  justify  the 
use  of  an  epi-spline  obtained  from  (FIPV)  as  an  approximation  of  an  actual  function  given  by  (FIP). 
Epi-convergence  of  (FIPU)  to  the  actual  problem  (FIP)  is  the  central  property.  We  denote  the  optimal 
values  of  (FIP)  and  (FIPU)  by  V  and  Vv,  respectively. 

3.15  Theorem  (convergence  of  minimizers  [2,  Theorem  2.5].)  Suppose  that  {(F I Pu)}„£]y  epi-converges 
to  (FIP),  sk  minimizes  (FIPUk),  k  G  IV,  and  d(sk,f)  — »•  0.  Then,  f  is  a  minimizer  of  (FIP)  and 
lim  VVk  =  V. 

We  next  give  a  sufficient  condition  for  epi-convergence  of  (FIPU)  to  the  actual  problem  (FIP).  We 
recall  that  i(>l/  converges  continuously  to  if  relative  to  F  if  for  every  /  G  F  and  sequence  sv  — >•  /,  with 
sv  G  F,  ifu(sv)  — >•  if(f).  Let  int  S  denote  the  interior  of  a  set  S. 

3.16  Theorem  (sufficient  condition  for  epi-convergence.)  If  ifv  converges  continuously  to  if  relative 
to  F,  UI,eiv51'  is  dense  in  F ,  and  Fu  converge 6  to  F  =  cl(intF),  then  {(FIPu)}l,£]y  epi-converges  to 
(FIP). 

Proof.  We  here  follow  the  argument  of  Theorem  2.5  of  [30],  but  include  it  for  completeness.  We 
first  consider  a)  in  Definition  3.14.  For  {su}U£Ki  with  K  an  infinite  subset  of  IV,  sv  G  Fu  <lSu,  and 

6In  the  sense  of  Painleve-Kuratowski;  see  footnote  of  §2  and  [26,  Chapter  4]. 


11 


d(sv ,  /)  — >•  0,  we  immediately  find  that  /  £  F  from  the  assumption  that  F"  — >•  F.  In  view  of  the 
continuous  convergence  of  ipv  to  i/j,  we  establish  part  a).  Second,  we  consider  part  b)  and  let  f  €  F  be 
arbitrary.  Since  F  =  cl(intF),  there  exists  a  sequence  {fp}p^N  C  int  F  such  that  d(fp,f)  — )•  0.  For 
each  [j,,  the  facts  that  U U£nSu  is  dense  in  F,  Fu  converges  to  F,  and  belongs  to  an  open  subset 
of  F  imply  that  there  exists  a  sequence  C  Fu  n  Sv  such  that  d(sp’u,  /M)  —t  0  as  v  — >•  oo. 

Consequently,  we  can  construct  a  sequence  jy  such  that  z/*+1  >  v*,  u*  >  max{^,  z^+i},  and 

d(sp,,v ,  fp)  <  l//i  for  all  z/  >  v*  and  p.  For  arbitrary  £  >  0,  there  exists  therefore  an  integer  fiQ  >  2/e 
such  that  for  all  u  >  v*  and  p  >  po, 

d(rj)<e/ 2  and  d(s^,  n  <  e/2. 

We  construct  the  sequence  C  F1'  n  Su  by  setting 

sv  =  s^u  with  p  satisfying  v  €  {zz*_i  +  1,  z/*_i  +  2, ...,  v*}. 

Then  for  every  v  >  v*lQ  and  some  p„  >  po, 

d(sv,  f)  =  d(s^,  f)  <  d(s D  +  <  e/2  +  e/2  =  e. 

Consequently,  d(su,f)  — >•  0  and  ^u{sv)  — >•  by  continuous  convergence,  which  establish  part  b).D 

We  note  that  the  constraint  qualification  F  =  cl(intF),  i.e.,  F  is  solid,  avoids  “isolated”  feasible  points 
that  cannot  easily  be  approximated. 


3.3  Estimating  Epi-Distances 

We  next  provide  estimates  of  epi-distances,  especially  between  epi-splines,  and  make  connections  be¬ 
tween  epi-convergence  and  other  modes  of  convergence.  It  is  immediately  clear  from  [26,  Proposition 
7.15]  that  uniform  convergence  implies  epi-convergence,  which  we  utilize  repeatedly  below.  However, 
the  converse  fails  as  illustrated  below.  In  fact,  epi-convergence  can  be  viewed  as  a  one-sided  convergence. 

3.17  Theorem  (epi-distance  estimates)  For  s,s'  €  e-spl^(F),  with  1Z  =  {Rk}^=l,  one  has  for  any 
P>  o, 

(i)  d(s,  s')  <  || s  —  s^loo  and  dp(s,  s')  <  ||s  —  s'Hoo 

(ii) d(s,s')<  max  sup  |s(x)  —  s\x)\  and  dp(s,s')<  max  sup  \s(x)  —  sr(a;)|. 

k=l,...,N  xeRk  k=l,...,N  xeRk 

Proof:  We  first  consider  dp(s,s').  For  any  x  =  (x,xq)  €  plB  C  Mn+1,  d(x,epis)  <  d(x,  epis')  +  ||s  — 
s'Hoo-  Reversing  the  roles  of  s  and  s'  yields  that  \d(x,  epis)  —  d(x,  epis')|  <  ||s  —  s'Hoo-  Consequently, 
dp(s ,  s')  <  || s  —  s'Hoo-  Since  J0°°  e~pdp  =  1,  we  also  conclude  that  the  first  part  of  item  (i)  holds.  From 
(i)  it  is  clear  that  item  (ii)  would  hold  with  R replaced  by  cl  R^-  The  “closure”  is  superfluous  for 
the  following  reason.  For  every  €  Mn  there  exists  a  sequence  xv  and  k*  €  {1,2,  such 
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that  xv  G  Rk*  for  all  v  and  lim „s'(xu)  =  s'(x°)  by  the  definition  of  epi-splines  and  openness  of  Rk, 
k  =  1,  Since  s(x°)  =  lim  inf x/_>xo  s(x')  and  s  is  continuous  on  Rk* ,  it  then  follows  that 

s(x°)  —  ^(x0)  <  lim^  s{xv)  —  lim,,  s'{xv)  <  sup  |s(x)  —  s^x)!- 

Reversing  the  roles  of  s  and  s' ,  yields  that 

s(x°)  —  /(x0)  >  —  sup  |s(x)  —  ^(x)). 

x&Rf.* 


Consequently, 


|s(x°)  —  ^(x0)!  <  max  sup  |s(x)  —  s7 (x) | , 
k=l,...,N  xsRk 


which,  because  x°  G  lRn  is  arbitrary,  completes  the  proof  of  item  (ii)  after  again  using  the  fact  that 

r  e~pdp = l  □ 

We  observe  that  since  s  —  s'  is  also  a  polynomial  on  Rk,  say  q  G  polyp(J?n), 


sup  |s(x)  —  s'(x)\  =  max  <  sup  q{x),—  inf  q(x) 

xGRk  I  xGclRk  xGclRfc 

In  general,  optimization  of  polynomial  over  an  arbitrary  closed  set  is  difficult,  but  many  special,  more 
tractable  cases  exist,  especially  in  practically  important  situations  with  low  degree  and  partitions  con¬ 
sisting  of  boxes. 


By  Theorem  3.17,  a  sequence  {sl'}£T1  C  e-spl^(7£)  converging  to  s°  €  e-spl^(7£)  in  the  L°°-norm,  i.e., 
\\s’/  —  s°||oo  — >•  0,  also  converges  in  the  epi-distance,  i.e.,  d(su,s°)  — >•  0.  The  converse  fails,  however, 
as  the  following  counterexample  on  1R  demonstrates.  For  1Z  =  {(— oo,  0),  (0,  oo)}  and  v  G  IN,  let 
sv  G  e-spl]  (77.)  be  su(x)  =  0  if  x  <  0  and  s^x)  =  x/v  otherwise.  We  also  define  s°(x)  =  0  for  all 
x  G  JR.  Clearly,  ||s^  —  =  oo  for  all  v  G  IN.  However,  for  any  p  >  0,  dp(su,  s°)  <  p/v  and  therefore 

d(su,s°)  <  (1/zz)  J0°°  pe~pdp  =  1/za  Consequently,  d(su,s°)  — >  0,  but  Hs1'  —  s°||oo  -/>  0. 

Further  estimates  of  the  epi-distance  are  available  through  a  supporting  quantity  defined  next,  which 
as  d  and  dp,  fully  characterize  epi-convergence;  see  [26,  Theorem  7.58].  We  let  for  any  f,g  :  JRn  — >•  JR, 

dp(f,  g )  =  inf {77  >  0  |  epi  /  n  pJB  C  epi  g  +  qJB,  epi  g  fl  pJB  C  epi  /  +  pJB}. 

As  dp,  dp  is  closely  related  to  the  Pompeiu-Hausdorff  distance;  see  [26,  Chapters  4  and  7].  Moreover, 
for  f,g  convex  with  /( 0)  <  0  and  g( 0)  <  0,  dp(f,g)  =  dp(f,g )  for  any  p  >  0  by  [26,  Exercise  7.60]. 
The  same  exercise  also  provides  the  following  more  general  estimates. 

3.18  Proposition  For  f,g  :  JRn  — >•  JR  not  identical  to  00,  the  following  hold  with  df  =  d(0,epi/)  and 
dg  =  <7(0,  epi  <7): 


13 


0)  dp(f,g )  <  dp(f,g )  <  dp>{f,g),  when  p'  >  2p  +  max{df,  dg} 

(ii)  d(f,g)  >  (1  -  e~p)\df  -  dg\  +  e~Pdp(f,g) 

(hi)  d(f,  g)  <  (1  -  e~p)dp(f,  g)  +  e~p(max{df,dg}  +  p+l). 

Computation  of  the  epi-distance  by  combining  results  for  each  set  in  a  partition  is  supported  by  the 
next  result,  which  especially  is  useful  if  the  diameter 

diam(i?fc)  :=  sup  \\x  —  x'\\ 

a;,®' Gel  Rk 


of  Rk  is  small.  For  s  €  e-spl^(7£),  we  adopt  the  notation 

sk  :=  s  +  6c\Rk,  where  dciRk  (x)  =  0  if  x  €  c\Rk  and  oo  otherwise. 

3.19  Theorem  (additional  epi-distance  estimates)  For  s,s'  €  e-spl^(7£),  with  1Z  =  {Rk}k Li>  one  has 

(i)  dp(s,s')  <  max  dp(sk,s'k ) 

k=l,...,N 

(ii)  For  any  k  =  1, TV,  dp{sk,  s'k )  =  0  if  epi  S&  fl  pJB  —  epi  n  plB  =  0,  and  otherwise 


ULp 

max 


(sk,s'k)/V2  < 

diam(l?;c),  inf  s' (x)  —  max  <  min  s(.x),—  p  >  ,  inf  s(x) 

xGclR/c  [x£cl  Rf-HpR  J  xGclRk 


—  max 


<  min  s'(x),—p 

[xec\RknPB 


Proof:  First  consider  part  (i).  Since  for  all  k  =  1, ...,  N, 

epi  Sk  fl  plB  C  epi  s'k  +  dp(sk ,  s'k)lB , 


we  also  have  that 


t  N 

u 

vA:=  I 


N 


epi  sk 


Fl  plB  C  I  |^J  epi ; 


^fc=i 


+  max 
k=l,...,N 


dp(sk,  sk)lB. 


A  similar  results  with  the  roles  of  sk  and  s'k  reversed  and  the  fact  that  epi  s  =  epi  sk  and  likewise 
for  s'  yield  the  conclusion.  Second  consider  part  (ii).  The  first  case  follows  trivially  from  the  definition 
of  dp.  The  more  involved  case  relies  on  [26,  Proposition  7.61]  from  which  we  deduce  that  dp(sk,  s'k)/V 2 
is  no  larger  than  the  infimum  of  all  i ]  >  diam(i?fc)  such  that  for  all  x  €  plB  n  cli?^, 


inf  s(x')  <  maxjsTx),  —  p}  +  g  and  also  with  the  roles  of  s  and  s'  reversed, 
z'ecl  Rk 

Consequently,  g  must  satisfy 

inf  six' )  —  max  <  min  s' (x ),  —  p  >  <  ??  and  also  with  the  roles  of  s  and  s'  reversed, 

x'ed  Rk  (xeclRfcnpB  J 

and  the  conclusion  follows.  □ 
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We  again  observe  that  optimizing  polynomials  in  general  is  challenging,  but  the  formula  in  Theorem 
3.19(h)  requires  only  minimization,  in  contrast  to  the  one  in  Theorem  3.17(h)  where  both  minimization 
and  maximization  are  needed,  and  therefore  in  the  case  of  convex  Sk  and  s'k  as  well  as  convex  is 
easily  implemented. 

We  end  this  subsection  with  a  result  that  highlights  the  connections  between  epi-splines  and  their 
representation  by  a  finite  number  of  parameters.  We  start  with  an  intermediate  result. 

3.20  Proposition  (functional  convergence  of  polynomials).  Suppose  that  qv  £  poly p(Mn),  v  £  INq, 
and  c  =  (ci,C2, ... ,cUp )  is  a  basis  for  such  polynomials,  i.e.,  there  exists  a  unique  a?  £  lRnp  such  that 
qu  =  (c^),^),  v  ^  W0.  Then, 

d(qu ,  q°)  — »•  0  av  — >  a0. 

Proof.  Since 

l^-90|  =  l(c(-),al/-a0)l<l|c(-)lll|a"-«0|| 

and  c  is  continuous  (in  fact  polynomial),  convergence  av  — >  a0  implies  uniform  convergence  of  qu  to  q° 
on  any  compact  subset  of  Mn.  Using  Proposition  2.1,  a  standard  argument  shows  that  qv  epi-converges 
to  q°  and  therefore  also  d(qu,q°)  — >•  0.  Next,  we  consider  the  converse.  Suppose  for  the  sake  of  a 
contradiction  that  d(qI/,q0)  — >•  0,  but  there  exists  a  p  >  0  and  an  infinite  subset  Noo  of  IN o  such  that 
\\au  —  a0 1 1  >  p  for  all  u  £  N, \yo.  Let  x1,  ...,x71p  £  lRn  be  a  collection  of  distinct  points  such  that  every 
polynomial  q  £  polyp(J?M)  is  uniquely  defined  by  the  solution  of  the  system  C°a  =  b°(q),  where  C°  is 
the  np-by-np  matrix  with  rows  c(xl),  i  =  1,  and  b°(q)  is  the  transpose  of  (q(x1),  ...,q(xnp)).  That 

is,  q  =  (c(-),a),  where  a  is  the  solution  of  C°a  =  b°(q).  Let  eM\0,  as  p  — >•  oo,  and  the  open  balls 

B?  =  {(x,x0)  £  Mn+1  |  ||(x,*0)  -  (*<,g°(*<))||  < 

For  each  p  and  i,  it  follows  from  the  “hit-and-miss  criterion”  of  [26,  Proposition  4.5(a)]  that  there  exists 
a  v?  such  that  for  all  v  >  uf  epi qur\B^  /  0.  Moreover,  by  [26,  Proposition  4.5(b)],  there  exists  a  z/q  and 
a  compact  set  S  C  lRn  containing  x1,  ...,x71p  such  that  for  all  u  >  qv  >  q°  —  efJ'/ 2  on  S.  Consequently, 
with  z/^  =  max  [ zzg  ,  ...,  uHp  ],  the  graph  of  qv  intersects  all  the  balls  B-1,  i  =  1, ...  ,np,  for  all  p  and 

v  >  zzffi  Let  (xl,u,tl,qu(x'l,u,fJ’)),  i  =  1, . . . ,  np,  be  such  intersection  points,  i.e.,  (aC1^,  qu  (xl,u,IJ'))  £  Btf  for 
i  =  1, . . .  ,np,p  €  IN,  and  n  >  zzffi  Let  be  the  np-by-np  matrix  with  rows  c(x v',/i),  i  =  1,  ...,np, 
and  bv'^(qu)  be  the  transpose  of  (qv( ...,qu(xnp’u’f1)).  For  sufficiently  large  p,  the  unique  solution 
au,fl  of  Cu,tJ,a  =  bu,^{qv)  coincides  with  the  coefficients  av  of  qv  for  u  >  zz^,  p  £  IN.  Moreover,  and 
a0  are  the  unique  optimal  solutions  of 

min  HC^a  -  bv’» (qu)\\2  and  min  \\C°a  -  b°(q°) ||2, 

a  a 

respectively.  Since  |  0,  as  p  — >  oo,  Cu^  — >•  C°  and  bv^(qu)  — >•  b°(q°),  as  p  ->  oo  and  zz  >  u11, 
the  objective  function  of  the  first  problem  epi-converges  to  that  of  the  second  problem  as  p  — >•  oo  and 
v  >  zzffi  Theorem  7.31  of  [26]  then  implies  that  av,,Ji  a0  as  p  — >•  oo,  zz  >  zzffi  Consequently,  there 
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exists  a  p,  such  that  \\au,fl  —  a°||  <  p  for  all  p  >  p,  v  >  Since  a =  a?  for  v  >  ,  we  have  reached 

a  contradiction.  □ 

3.21  Theorem  (convergence  of  the  parameters).  Suppose  that  su  €  e-spl(](77),  with  IZ  =  {Rk}^=\, 
v  €  IN o,  and  c  =  (ci,  C2, ...,  cnp)  is  a  basis  for  polynomials  in  polyp  (IPn) ,  i.e.,  for  unique  ak,u  €  lRnp, 

su(x )  =  (c{x),ak’u%  when  x  €  i?fc. 


Then, 


1,0 


,JV,0 


s°  uniformly  on  compact  sets  S  C 


0. 


Proof.  Since  polynomials  of  total  order  p  convergence  uniformly  on  compact  sets  if  and  only  if  their 
coefficient  converges,  we  obtain  the  first  implications.  In  view  of  Proposition  3.20  and  the  fact  that 
uniform  convergence  implies  epi-convergence  [26,  Proposition  7.15],  the  second  implication  holds.  □ 


4  Composite  Epi-Spline 

Probability  density  estimation  [29,  34]  is  one  of  our  major  incentives  for  considering  problems  of  the 
form  ( FIP )  and  constructing  evolving  approximations  ( FIPV ).  A  density  is  a  nonnegative  function 
that  sums  up  to  1  and  an  estimate  is  chosen  so  as  to  minimize  some  appropriate  criterion;  for  further 
details  see  §6.2,  [29],  and  references  therein.  In  addition,  many  “standard”  densities  belong  to  an  ex¬ 
ponential  family  [3],  which  lead  us  to  building  density  estimates  in  terms  of  an  exponential  function 
composed  with  an  epi-spline,  which  are  nonnegative  automatically,  rather  than  epi-splines.  In  other 
cases,  a  nonlinear  transformation  of  this  kind  may  improve  conditioning  of  (FIP)  and  therefore  facil¬ 
itate  its  numerical  implementation.  On  a  theoretical  level  the  results  for  such  composite  epi-splines , 
including  the  formulation  of  (FIP)  in  terms  of  composite  epi-splines,  follow  rather  straightforwardly 
from  those  in  §2  and  §3.  Still,  for  easy  reference  and  a  better  understanding  of  the  specific  properties, 
it  is  useful  to  record  the  central  results. 

We  start  by  defining  the  set  of  composite  epi-splines  corresponding  to  a  function  9  :  IR  — >  1R. 

4.1  Definition  (composite  epi-splines).  A  composite  epi-spline  h  :  Mn  — >  IR  of  order  p  €  IN q,  with 
partition  IZ  =  {i?fc}^=1  of  IRn  and  function  9  :  IR  IR,  is  a  function 

h  =  9  o  s,  where  s  €  e-spl([(77). 

The  family  of  all  such  composite  epi-splines  is  denoted  by  c-spl(](77,  9). 

Composite  epi-splines  are  generally  not  lsc  and  may  be  upper  semi-continuous  (use)  as  we  see  next.  We 
denote  by  use- fens (J?n)  the  space  of  all  use  /  :  IRn  — >•  IR,  excluding  /  =  —  oo.  After  observing  that  the 
hypograph  of  a  function  /,  hypo  /  =  |(x,  xq)  |  f(x)  >  xq}  is  just  a  mirror  image  of  the  epigraph  of  — /, 
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one  can  mimic  the  definitions  and  constructions  described  for  lsc  functions  to  set  up  the  hypo-distance 
^hypo(/,  9)  =  d(—f,  — g ),  between  any  two  functions  /  and  g  and  generate  the  hypo-topology  which  again 
makes  (usc-fcns,  d)  a  Polish  space.  A  sequence  of  functions  fv  hypo-converge  to  /  if  —fv  epi-converge 
to  — /;  see  [26,  Chapters  4  Sz  7]  for  a  broader  treatment. 

4.2  Proposition  For  a  continuous  9  :  M—t  IR,  the  following  hold: 

(i)  If  9  is  increasing,  then  c-spl^(7£,  9)  C  lsc-fcns(Mn). 

(ii)  If  9  is  decreasing,  then  c-spl^(7£,  9)  C  usc-fcns(lRn). 

Proof:  These  results  follow  as  direct  consequences  of  definitions  of  lsc  and  use  functions.  □ 

4.3  Theorem  (dense  approximations  by  composite  epi-splines).  Under  the  assumption  of  Theorem 
3.5  and  a  continuous  9  :  IR . M,  the  following  hold: 

(i)  If  9  is  increasing  and  extended  to  IR  hy  setting  9 (00)  =  sup  9  and  9(— 00)  =  inf  9,  then 

OO 

|^J  c-spl^ft1',  9)  is  dense  in  {9  o  f  \  f  e  lsc-fcns(Mn)}  under  the  epi-topology. 

V=1 


(ii)  If  9  is  decreasing  and  extended  to  IR  by  setting  9( 00)  =  inf  9  and  9(— 00)  =  sup#,  then 

OO 

|^J  c-spl Cl  (IZu,9)  is  dense  in  {9  o  f  \  f  e  usc-fcns(IRn)}  under  the  hypo-topology. 

v=\ 

Proof:  By  Theorem  3.5,  for  every  s°  €  lsc-fcns(J?n)  there  exists  a  sequence  with  su  € 

e-spl^(7 Zu)  for  all  v,  such  that  d(su,s°)  — >•  0.  By  Exercise  7.8  of  [26],  it  then  follows  that  9  o  sv  epi- 
converges  to  9  o  s°  and  the  first  conclusion  follows.  The  second  result  is  a  consequence  of  an  identical 
argument  under  a  sign  change.  □ 


5  Extrinsic  information 

Information  about  the  nature  of  solutions  of  ( FIP )  may  justify  the  consideration  of  subsets  of  lsc- fens (IRn) 
as  defined  through  F  and  T  and  likewise  subsets  of  e-splj[  (77) .  We  here  provide  a  few  examples  of  such 
information  and  their  implementation  as  constraints  in  (FIPV). 

Domain.  External  information  may  indicate  that  the  (effective)  domain  of  solution  functions  of  (FIP) 
is  a  closed,  strict  subset  B  of  Mn .  Then,  a  partition  of  B  instead  of  Mn  by  selecting  1Z  =  {Rk}^=i 
such  that  cl  Rk  =  B  avoids  wasting  computational  effort  on  uninteresting  parts  of  lRn.  In  general, 
the  specific  choice  of  IZ  is  guided  by  the  flexibility  required,  with  a  small  number  of  subsets  needed 
when  solutions  of  (FIP)  are  (nearly)  polynomials  of  order  p,  and  by  implementation  issues,  which  can 
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become  substantial  if  intricate  subsets  are  combined  with  constraints  of  the  type  given  in  the  remainder 
of  this  section  as  well  as  other. 


Continuity.  There  may  be  a  need  to  limit  the  consideration  to  solution  functions  that  are  continuous 
on  the  whole  or  parts  of  lRn.  For  a  partition  1Z  =  {Rk}k=i>  an  epi-spline  defined  in  terms  of  the 
polynomials  qk  €  poly p(lRn),  k  =  1,2, N,  is  continuous  on  the  boundary  between  Rk  and  Rp  k  /  l, 
if 


Qk{x)  =  qi{x)  for  all  x  €  c\Rk  fl  cl  Ri. 


(1) 


Certainly,  continuity  in  the  case  of  p  =  0  implies  that  q =  qi ,  but  continuity  for  intricate  R k  and 
Ri  can  also  force  q =  qi,  which  may  not  always  be  desirable.  Less  trivial  examples  are  provided 
by  partitions  consisting  of  subsets  Rk,  k  =  that  are  defined  by  a  finite  number  of  unions  and 

intersections  of  halfspaces.  We  refer  to  such  a  subset  as  a  finite  polytope.  The  next  proposition  supports 
the  implementation  of  constraints  with  this  structure. 


5.1  Proposition  Suppose  that  Rk  and  Ri  are  finite  polytopes  with  a  common  facet  defined  by  (c,x)  = 
b,  and  qk,qi  £  poly p(lRn).  Then,  the  constraints 

Qk(x)  =  qi(x)  for  all  x  €.  cl  Rk  Fl  cl  Ri  with  (c,  x)  =  b  (2) 

are  equivalent  to  (n  —  1  +  p)\/((n  —  l)!p!)  equality  constraints  on  the  coefficients  of  qk  and  qp 

Proof:  The  constraint  qk(x)  =  qi(x)  is  equivalent  to  q(x)  =  0,  with  q  =  qk  —  qi  €  poly p(lRn).  Since  a 
polynomial  in  polyp(K")  vanishes  on  an  infinite  number  of  points  if  and  only  if  all  its  coefficients  are 
zeros,  the  conclusion  follows  after  observing  that  (n  —  1  +p)!/((n  —  l)!p!)  is  the  number  of  coefficients 
for  polynomials  in  poly p(lRn~1),  where  the  dimensional  reduction  is  caused  by  the  restriction  to  the 
facet.  □ 

Naturally,  the  expressions  for  the  finite  number  of  equality  constraints  in  Proposition  5.1  can  get  involved 
and  we  here  provide  specifics  only  for  the  simple,  but  useful  case  n  =  p  =  2  with  a  partition  consisting 
of  rectangles  aligned  with  the  coordinate  axes.  We  let  qk{x)  =  a,Q+a\x\  +  a^X2 + a\2xiX2+ a\1x\+ 
and  similarly  for  qi.  For  a  facet  defined  by  Xi  =  6,  i  =  1  or  2,  we  obtain  the  three  constraints 

ao~ao  +  (ai  ~  ai)b  +  (<4  -  au)b2  =  0 

aj  ~  ai  (ai2  ~  ai2)b  =  0 
ajj  ~  ajj  ~  0 

with  j  =  2  when  i  =  1  and  j  =  1  when  i  =  2. 


Continuous  Differentiability.  Partial  derivatives  of  a  polynomial  in  poly p(fRn),  p  €  JV,  is  a  poly¬ 
nomial  in  polyp^1(^?n)  and  continuous  differentiability  simply  requires  continuity  of  those  derivatives. 
Consequently,  we  ensure  that  an  epi-spline  is  continuously  differentiable  by  imposing  the  conditions 
of  the  previous  paragraph  for  each  partial  derivative.  For  the  example  with  n  =  p  =  2,  each  facet 
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requires  two  constraints  per  partial  derivative  for  a  total  of  four  constraints.  Obviously,  higher  order 
differentiability  follows  the  same  pattern. 

Convexity.  The  convexity  of  an  epi-spline,  in  general,  requires  the  convexity  of  the  polynomials  on  R\ , 
...,  R]y,  and  also  the  proper  behavior  on  the  boundary  between  such  subsets.  The  following  provides  a 
supporting  result,  which  allows  for  simplifications  if  an  epi-spline  is  continuously  differentiable. 

5.2  Proposition  Suppose  that  s  £  e-spl^(7£),  with  TZ  =  {Rk}k=i>  JS  defined  in  terms  of  the  polyno¬ 
mials  qk  £  poly p(lRn),  k  =  1,  Then,  s  is  convex  if 

(i)  qk  are  convex  on  convex  sets  Rk,  k  =  1, ...,  N, 

(ii)  for  all  x  £  cl  Rk  n  cl  Ri,  y  £  Rk,  and  fc,  Z  £  {1,2, ...,  N}, 

qk(x)  =  qi(x )  and  (Vqk(x)  -  Vqi(x),y-  x)  >  0. 

If  s  is  continuously  differentiable,  then  item  (ii )  is  satisfied  automatically. 

Proof:  Since  qk  is  continuous  on  Mn,  the  convexity  on  Rk  implies  convexity  on  cliZ^.  Consequently, 
for  x  £  cl  Rk  n  cl  Ri ,  y  £  Rk, 

Qk{y)  >  qk(x)  +  (Vqk(x),y-  x) 

=  qi(x )  +  (' Vqk(x )  -  Vqi(x),y-  x)  +  (Vqt(x),y  -  x) 

>  qi(x)  +  (Vqi(x),y  —  x). 

We  next  establish  the  convexity  of  s  by  showing  that  for  x  €  cl  Ri  and  y  £  c\Rk,  x  /  y  and  k  /  l, 

qk(y )  >  qi(x)  +  (' Vqi(x ),  y  -  x). 

We  start  with  the  case  when  the  line  segment  {z  €  lRn  \  z  =  a(y  —  x)  +  x,a  £  (0, 1)}  intersects  cl  Ri 
and  cl  Rk,  and  not  cli?//  for  l'  /  /,  k.  Then,  there  exists  an  d  £  (0, 1)  such  that  z  =  d(y  —  x)  +  x  is  in 
both  cl  Ri  and  elf?*,.  From  above  and  the  convexity  of  qj,  we  obtain  that 

qk(y)  >  qi{z)  +  (Vqi(z),  y-  z) 

>  qi(x)  +  (Vqi(x),z  -  x)  +  (Vqi(z),  y  -  z). 

The  convexity  of  qi  also  ensures  that 

<V qi(z),y  -  z)  -  (Vqi(x),y  —  z) 

=  (Vqi(z)  -  Vqi(x),y  -  z) 

=(l/d-  T)(Vqi(z)  —  Vqi(x),  z  —  x)  >  0, 
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because  (1/a  —  1)  >  0.  Consequently, 

Qk(y)  >  Ql{x )  +  (Vqi(x),z-  x )  +  (Vqi(x),y  —  z) 

=  qi{x)  +  ( Vqi(x),y  -  x), 

which  establishes  the  condition  for  the  stated  case.  The  general  case  is  established  by  applying  the 
above  arguments  repeatedly  at  each  of  the  finite  number  of  points  on  the  line  segment  {z  €  Mn  \  z  = 
a(y  —  x)  +  x,  a  €  (0, 1)}  where  s  is  not  continuously  differentiable. 

We  next  turn  to  the  second  claim.  If  s  is  continuously  differentiable,  then  qk(x)  =  qi(x)  and 
Vqk{x )  =  S7qi(x)  for  all  x  €  clRk  H  cl  R[  and  k,l  €  {1,  2, ...,  IV},  which  confirms  that  the  second 
conclusion  holds.  □ 

We  observe  that  item  (ii)  in  Proposition  5.2  relates  to  the  monotonicity  of  subgradients  of  s. 

Monotonicity.  We  say  that  a  function  /  :  Mn  — >•  JR  is  nondecreasing  if 

f(x)  <  f(y)  whenever  x  <  y, 

where  the  last  inequality  is  interpreted  componentwise.  For  an  epi-spline  s  €  e-spl^(7£),  with  TZ  = 
{Rk}k= i  and  polynomials  qk  €  poly p(lRn),  k  =  1  to  be  nondecreasing,  it  is  obviously  needed 

that  qk(x)  <  qk{y )  for  x,y  €  Rk  with  x  <  y.  Since  q f.  is  differentiable,  this  is  ensured  by  V%(x)  >  0 
for  x  €  Rk-  If  p  =  2  and  Rk  is  a  finite  polytope,  then  Vqk  is  affine  and  it  suffices  to  impose  Vqk(x)  >  0 
for  vertices  x  of  cl  Rk  ■  It  is  clear  that  under  continuity,  boundary  points  are  immaterial: 

5.3  Proposition  If  s  €  e-spl^(7£),  with  TZ  =  {Rk}k=i’  JS  defined  in  terms  of  the  polynomials  qk  € 

poly p(TRn),  k  =  1  and  is  continuous,  then  s  is  nondecreasing  whenever  qk,  k  =  1  are 

nondecreasing. 

In  general,  however,  an  epi-spline  may  not  be  nondecreasing  even  if  qk,  k  =  1,  are  nondecreasing. 

If  Rk,  k  =  1, ...:,  N,  are  finite  polytopes  consisting  of  boxes,  i.e.,  sets  of  the  form  Rk  =  {x  €  lRn  \  Ik  < 
x  <  Uk},  then  the  following  proposition  provides  guidance,  where  we  say  that  Rk  precedes  Ri  if  there 
exist  x  €  Rk  and  y  €  Ri  with  x  <  y. 

5.4  Proposition  If  s  €  e-spl}((7£),  with  TZ  =  {Rk}k=i  consisting  of  boxes,  is  defined  in  terms  of  the 
polynomials  qk  €  polyp(H"),  k  =  1, N ,  then  s  is  nondecreasing  if 

(i)  qk,  k  =  1, ...,  N,  are  nondecreasing 

(ii )  qk  (x)  <  qi  (x)  for  x  €  cl  Rk  n  cl  Ri  and  Rk  preceding  R[ . 

A  nonincreasing  epi-spline  is  treated  similarly  with  the  appropriate  reversal  of  inequalities. 

Bounds.  The  nonnegative  of  an  epi-spline  requires  conditions  that  ensure  the  nonnegativity  of  poly¬ 
nomials,  which  except  for  orders  0  and  1  are  nontrivial.  A  composition,  as  discussed  in  §4,  constructed 
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from  the  exponential  function  guarantees  nonnegativity  automatically.  Specifically,  a  problem  optimiz¬ 
ing  over  h  £  lsc-fcns (lRn),  h  >  0,  can  be  reformulated  as  one  over  /  £  lsc-fcns(^?n)  by  setting  h  =  e* . 
Since  /  =  — oo  is  included  in  /  £  lsc-fcns(.ff?n)  and  corresponds  to  h  =  0,  the  reformulation  is  equivalent. 
Consequently,  instead  of  approximating  the  function  h  of  the  original  problem  by  s  £  e-spl^(7£),  s  >  0, 
we  can  approximate  the  function  /  of  the  reformulated  problem  by  s  £  e-spl))  (7Z).  Nonpositivity  and 
other  bounds  are  treated  similarly. 

Log-concavity.  A  function  h  :  Mn  — >•  [0,  oo),  with  h  >  0  on  a  convex  set  X,  is  log-concave  on  X  if 
log  h  is  concave  on  X.  Again  a  composition  is  convenient.  A  problem  optimizing  over  h  £  lsc-fcns(Kn), 
h  >  0  and  log-concave  on  X,  can  be  reformulated  as  one  over  /  £  lsc-fcns(.ff?n),  /  concave  on  X,  by 
setting  h  =  e* .  The  concavity  constraint  is  then  handled  by  ensuring  convexity,  as  described  above,  of 
the  negative  of  the  corresponding  approximating  epi-spline. 

Integral  conditions.  Requirements  that  functions  /  £  lsc-fcns(K)  in  ( FIP )  should  satisfy7 

l  <  /  f(x)dx  <  u 

Jx 

for  an  open  set  X  C  Mn  is  handled  by  ensuring  that  an  epi-spline  given  by  polynomials  qk,  k  =  1, ...,  N, 
and  partition  1Z  =  {Rk}^=i  satisfies 

N  r 

l  <  /  qk(x)dx  <  u. 

k=l ^ 

Due  to  the  polynomial  forms,  the  integrals  are  here  easily  computed  analytically  whenever  the  descrip¬ 
tions  of  X  and  Rk  are  “simple.” 

Proximity.  Given  an  epi-spline  s°  £  e-spl^(T^),  with  1Z  =  {Rk}k=i  and  corresponding  polynomials 
q®  £  poly p(Mn),  k  =  1,  ...,1V,  applications  may  require  epi-splines  in  e-spl^(72.)  that  are  “close”  to  s°. 
Constraints  on  norms  between  s°  and  s  £  e-spl^(7£),  given  by  qk  £  poly p(lRn),  k  =  1, ...,  N ,  is  facilitated 
by  the  expression 


If  m  is  even,  then  the  right-most  integrals  are  easy  to  compute  analytically  as  the  integrand  is  polyno¬ 
mial  as  long  as  Rk  is  “simple.” 

Subgradient  bounds.  We  recall  the  notion  of  subgradients  of  a  function  /  :  JRU  — >•  1R,  where  we  need 
the  notation  xu  -f>-x  to  denote  a  sequence  xv  — >•  x  that  also  satisfies  f{xu)  — >  f(x).  A  comprehensive 

7Since  /  £  lsc-fcns(JR”),  the  level  sets  {x  €  R  \  f(x)  <  7}  are  closed.  Consequently,  /  is  measurable  and  for 
open  sets  A'  C  Mn,  the  integrals  Jx  f+(x)dx  and  f  f-(x)dx,  with  /+  =  max{0, /}  and  /_  =  max{0,  — /},  are  well- 
defined,  but  possibly  infinite.  With  the  usual  conventions  00  —  a  =  00,  a  —  00  =  —00,  and  00  —  00  =  00,  jx  f(x)dx  = 
fx  f+(x)dx  —  fx  f-{x)dx  is  therefore  well-defined. 
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treatment  of  the  topic  is  found  in  [26,  Chapter  8].  For  a  function  /  :  Mn  — >  ]R  and  v,  x  G  Mn  with  /(x) 
finite,  we  say  that 

(i)  v  is  a  regular  subgradient  of  /  at  x,  if 

liminf  fM-fW-b’y-*)  >  0, 
y-*x,y^x  \\y  —  x\\ 

with  the  set  of  all  regular  subgradients  denoted  by  <9/(x); 

(ii)  v  is  a  subgradient  of  /  at  x  if  there  are  sequences  xu  and  vv  — >•  v,  vu  G  df(xu),  with  the  set 
of  all  subgradients  denoted  by  df(x). 

Connections  between  the  subgradients  are  summarized  next. 

5.5  Proposition  Suppose  that  s  G  e-spl^(7£)  and  x  G  Mn .  Then,  ds(x)  and  ds(x)  are  closed  subsets 
of  Mn ,  ds(x)  is  convex,  and 

Bs(x)  C  ds(x)  /  0, 

with  equality  holding  if  s  is  regular8. 

Proof:  The  only  part  that  requires  a  proof  is  the  claim  about  nonemptiness.  Let  TZ  =  {Rk}k=i  and  s 
be  defined  in  terms  of  eg,  G  poly p(fRn),  k  =  1, ...,  N.  For  given  x  €  Mn,  let  k  G  {1, ...,  N}  be  such  that 
s(x)  =  qk{x )  and  x  G  cl  Rk-  There  exists  a  sequence  xv  -A  x  with  xv  G  Rk ■  Since  Vqk(xu)  — >•  \7qk{x)  and 
{Vqk(x1')}  =  ds(xu),  V^fc(x)  €  ds(x)  and  ds(x)  is  therefore  nonempty.  The  remaining  claims  are  direct 
consequences  of  [26,  Theorem  8.6  and  Corollary  8.11]  due  to  the  finiteness  and  lower  semicontinuity  of 
s.  □ 

Constraints  on  subgradients  are  supported  by  the  following  results. 

5.6  Theorem  (subgradients)  Suppose  that  s  €  e-spl^(7£),  with  1Z  =  {Rk}k=i  consisting  of  Clarke 
regular 9  sets  cl  Rk,  k  =  1,2,  ...,1V,  is  continuous  and  defined  in  terms  of  qk  €  poly p(Mn),  k  =  1,  ...,1V. 
Then,  for  a  nonempty  and  closed  S  C  Mn , 

Vqk(x)  €  S  for  x  €  Rk,  k  =  1, ...,  N  ds(x)  C  5  for  x  €  . 

Proof:  Since  ds(x)  =  {Vg?j(x)}  for  x  €  Hfc,  the  implication  from  right  to  left  follows  trivially.  Next, 
we  consider  the  converse.  Let  x  €  and  <9s(x)  /  0.  If  d s(x)  =  {x},  i.e.,  is  a  singleton,  then  there 
exists  a  k  and  xu  — >•  x  with  x^  G  Rk  and  \7qk(xu)  — >•  v.  Since  \7qk(xu)  G  5  and  5  is  closed,  it  follows 
that  v  G  5.  Now,  suppose  that  ds(x)  is  not  a  singleton  and  let  v  G  ds(x)  be  arbitrary.  For  the  sake 

8We  recall  that  a  function  /  :  K"  — >  is  regular  if  for  every  a:  £  Rn ,  epi  /  is  Clarke  regular  at  (x,  /(*));  see  the  next 
footnote  and  [26,  Definitions  6.4,7.25].  In  particular,  if  /  is  convex,  then  it  is  regular. 

9  A  set  A  C  Rn  is  Clarke  regular  if  at  all  x  £  A,  (i)  A  n  B  is  closed  for  some  closed  neighborhood  Af  of  x  and  (ii)  every 
normal  vector  v  of  A  at  x  is  regular,  i.e.,  (v,  y  —  x)  <  o(\\y  —  x||)  for  y  £  A.  For  example,  if  A  is  locally  convex  at  x  for  all 
x  £  A,  then  A  is  regular;  see  [26,  Definitions  6.3  and  6.4]. 
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of  a  contradiction,  suppose  that  v  0  S.  Then,  since  c\Rk,  k  =  1, ...,  2,  are  Clarke  regular  there  exist 
k*  G  {1, ...,  N},  h  G  Mn,  5  >  0,  and  t*  >  0  such 

x  +  th  G  Rk *  and  s(x  +  th)  =  qk* (x  +  th)  for  all  t  G  [0,  t*] 


and 

(u  -  Vqk*(x),h)  >  5||/i||, 

where  the  last  inequality  is  a  consequence  from  the  fact  that  k*  can  be  selected  such  that  v  /  Vqk*(x). 
Let  e  G  (0,  (5)  and  t£  €  (0,  t*)  be  such  that 

| qk*(x  +  th)  -  qk*(x )  -  (Vqk*(x),th)\  <  et\\h\\  for  all  t  G  (0 ,te), 

which  follows  from  the  smoothness  of  qk*  ■  Using  these  facts,  we  obtain  that 


s(x)  +  (v,th)  =  s(x)  +  (v  —  Vqk*  (x),th)  +  (Vqk*(x),th) 

>  qk*  (x)  +  (Vqk*  (x),  th)  +  5t\\h\\ 

>  s(x  +  th)  +  (5  —  £)t||h|| 

for  all  t  G  (0  ,te).  Consequently, 

s(x  +  th)-s(x)-(v,th) 

- m - £“({“')<0 

for  all  t  G  (0 ,t£),  which  contradicts  the  assumption  that  v  is  a  regular  subgradient.  Since  the  situation 
with  ds(x)  =  0  is  trivial,  this  establishes  that  ds(x)  C  S  for  all  x  G  Mn.  Finally,  we  consider  v  G  ds(x) 
for  an  arbitrary  x  G  JRn.  Then,  by  definition,  there  exists  xv  — >•  x  and  vv  — >•  v,  with  vu  G  ds(xv).  Since 
ds(xu )  C  S  for  all  v,  v  G  S  due  to  the  closeness  of  S,  which  completes  the  proof.  □ 

In  view  of  the  preceding  results,  it  is  clear  that  applications  demanding  constraints  on  the  size  of 
subgradients  of  an  epi-spline  s  G  e-spl^’  (1Z)  are  satisfied  by  imposing  constraints  on  gradients  of  the 
corresponding  polynomials  qk  G  poly p(Mn)  on  Rk-  The  partial  derivatives  of  qk  would  then  be  in 
polyp-1(  Mn),  which  generally  would  require  an  infinite  number  of  constraints  to  ensure  the  inclusion 
in  a  set  for  all  x  G  Rk-  However,  if  p  <  2  and  Rk  is  a  finite  polytope,  then  it  suffices  to  enforce  the 
constraints  at  the  vertices  of  cl  R,k  ■ 


6  Applications 

We  illustrate  epi-splines  through  a  series  of  examples  arising  in  response  surface  construction  and  proba¬ 
bility  density  estimation.  Focusing  on  second-order  epi-splines  in  two,  occasionally  three,  dimensions,  it 
suffices  to  represent  polynomials  using  the  bases  (1,  x\,X2,  xf ,  xiX2X$)  and  (1,  aq,  X2,  X3,  x\ ,  X1X2,  X1X3, 

X2X3,  x|)  for  two  and  three  dimensions,  respectively.  Numerical  examples  in  one  dimension  are  found 
in  [30,  29,  28,  33].  We  only  consider  partitions  of  domains  of  interest  consisting  of  rectangles  of  equal 
size.  Though,  the  number  of  rectangles  varies.  CPLEX  (12.5.1.0)  and  CONOPT  (3.15L)  solve  resulting 
linear  and  nonlinear  programs,  respectively,  on  a  64-bit  Windows  7  laptop  running  at  2.60GHz,  with 
4GB  RAM,  after  they  are  formulated  in  GAMS  (24.1.3). 
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epi-spline  true  function 


3 


Figure  2:  Logarithm  example:  actual  function  (a),  randomly  generated  data  (b),  lsc  epi-spline  (c),  and 
continuous  epi-spline  (d). 
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(a)  (b) 

Figure  3:  Logarithm  example:  nondecreasing  continuous  epi-splines  (a)  and  also  with  s(0.1,  0.1)  <  —1.5 
(b). 

6.1  Response  Surface 

We  reconstruct  four  functions  based  on  observations  of  the  functions  at  a  finite  number  of  points  and 
extrinsic  information  about  the  smoothness  of  the  functions  and  other  factors. 

Logarithmic  function.  Suppose  that  fix)  =  log(xi  +  X2)  defined  on  the  domain  [0, 3] 2 ;  see  Figure 
2(a).  Relying  on  observed  function  values  at  25  randomly  generated  points  according  to  a  uniform 
distribution  on  [0,  3]2  (see  Figure  2(b))  and  a  partition  with  N  =  25  open  rectangles,  we  obtain  the 
least-squares  minimizing  epi-splines  of  Figures  2(c),  2(d),  and  3.  All  the  fits  achieve  essentially  a  zero 
error  at  the  25  data  points,  but  the  values  of  the  epi-splines  at  other  points  depend  on  the  extrinsic 
information  included.  Figure  2(c)  shows  the  fit  for  “unconstrained”  lsc  epi-splines.  The  fit  improves 
significantly  when  constrained  to  continuous  epi-splines  as  described  in  §5;  see  Figure  2(d).  We  achieve 
further  improvement  after  additionally  restricting  to  the  nondecreasing  epi-splines  (Figure  3(a))  and 
to  a  function  value  of  no  more  than  —1.5  at  (0.1,  0.1)  (Figure  3(b)).  The  actual  value  /( 0.1, 0.1)  =  —1.6. 

Inverse  function.  Suppose  that  f(x)  =  l/(x\X2)  defined  on  the  domain  [— 0.5,0. 5]2;  see  Figure  4(a) 
for  a  color  contour  plot,  with  red  and  blue  indicating  areas  with  high  and  low  function  values,  respec¬ 
tively,  and  white  corresponding  to  values  above  100  or  below  —100.  We  adopt  a  minimum  absolute 
deviation  criterion  and  unconstrained  lsc  epi-splines.  For  N  =  400  and  900  randomly  generated  data 
points  from  a  uniform  distribution  on  the  domain,  we  obtain  the  epi-spline  of  Figure  4(b).  The  total 
absolute  deviation  across  the  data  points  is  only  6.0,  but  errors  appear  elsewhere  as  the  “granular” 
picture  indicates.  Figure  5  shows  epi-splines  based  on  2500  data  points.  In  some  sense,  the  fit  improves 
as  indicated  by  Figure  5(a).  However,  total  absolute  deviation  increases  to  6- 105  as  the  partition  is  not 
fine  enough  to  capture  the  large  variation  of  /  near  the  origin.  The  situation  improves  with  a  finer  parti- 
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(a) 


(b) 


Figure  4:  Inverse  example:  actual  function  (a)  and  epi-spline  from  900  points  (b). 


tion  of  N  =  900,  which  results  in  an  essentially  negligible  total  absolute  deviation  of  1.4;  see  Figure  5(b). 

Trigonometric  function.  Suppose  that  f(x )  =  (cos(7rxi)  +  cos(7tx2))3  defined  on  [ — 3, 3]2 ;  see  Fig¬ 
ure  6(a).  We  rely  on  continuous  epi-splines  and  a  partition  with  N  =  400.  Based  on  900  uniformly 
distributed  data  points  and  a  max-deviation  criterion,  we  obtain  the  epi-spline  fit  of  Figure  6(b).  Max¬ 
imum  and  average  errors  across  the  data  points  are  0.415  and  0.313,  respectively.  Mean-square  and 
absolute  deviation  criteria  give  similar  results.  A  relaxation  of  the  continuity  requirement  results  in 
essentially  perfect  interpolation  at  the  expense  of  a  more  “rugged”  fit. 

We  also  examine  the  3-dimensional  function  f(x)  =  (cos(7rxi)+cos(7nc2)+cos(7TX3))3  defined  on  [—3,  3]3 
and  27,000  data  points  uniformly  generated.  Again  relying  on  continuous  epi-splines,  but  now  with  a 
partition  using  N  =  8000,  we  obtain  maximum  and  average  errors  over  the  data  points  of  5.16  and 
1.68,  respectively.  The  resulting  linear  program  consists  of  80,001  variables,  136,800  equality  con¬ 
straints  (including  redundancies),  and  54,000  inequality  constraints  (reduced  dual  LP  actually  solved 
has  27,086  rows,  111,402  columns)  and  solves  in  27  seconds  using  the  “barrier”  option  in  CPLEX.  A 
switch  to  an  absolute  deviation  criterion  yields  maximum  and  average  errors  of  7.74  and  1.49,  respec¬ 
tively,  and  solves  in  69  seconds.  When  relaxing  the  continuity  constraints,  errors  are  driven  to  near  zero. 

Sine  function.  Suppose  that  f(x)  =  sin(7r||x||)/(7r||x||)  for  x  /  0  and  /(x)  =  1  otherwise.  We  use  the 
least-squares  criterion  and  continuously  differentiable  epi-splines.  Figure  7  depicts  the  actual  function 
in  part  (a)  as  well  as  epi-splines  estimates  based  onJV  =  400  and  900  randomly  generated  data  points 
from  a  uniform  distribution  on  [—5,5]  x  [—5,5]  in  part  (b)  and  N  =  225  and  600  points  in  part  (c). 
The  mean-squared  error  in  both  cases  is  0.25. 
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(a)  (b) 

Figure  5:  Inverse  example:  using  2500  points,  epi-splines  with  N  =  400  (a)  and  N  =  900  (b). 


(a)  (b) 

Figure  6:  Trigonometric  example:  actual  function  (a)  and  epi-spline  (b). 


(a)  (b)  (c) 

Figure  7:  Sine  example:  actual  function  (a)  and  epi-spline  from  900  (b)  and  600  (c)  points. 
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6.2  Probability  Density  Estimation 


Probability  density  estimation  is  another  arena  where  epi-splines  show  promise.  We  refer  to  [29]  for  an 
in-depth  study  of  one-dimensional  density  estimation.  We  here  concentrate  on  two  dimensions.  Since 
densities  are  nonnegative  functions,  we  rely  on  composite  epi-splines  c-spl|(7£,  exp),  i.e. ,  exponential 
epi-splines  of  the  form  h  =  e~s ,  with  s  €  e-spl^T?.). 


Figure  8:  Normal  Example:  actual  density  (a)  and  continuously  differentiable  exponential  epi-spline 
for  sample  size  100  (b). 


For  a  sample  X1,  X2,  ...,XU  that  is  independently  and  identically  distributed  as  a  “actual  distribution,” 
it  is  well-known  (see  for  example  [29])  that  a  constrained  maximum  likelihood  estimator  is  an  optimal 
solution  of 

1  .P r 

max  —  n  f(Xl)ltu  such  that  /  >  0,  /  f(x)dx  =  1,  /  £  J, 

V  i= 1  ^ 

where  T  is  an  appropriately  selected  space  of  functions  on  Mn .  Passing  to  exponential  epi-splines,  we 
arrive  after  equivalently  maximizing  the  logarithm  of  the  objective  function  at  an  approximate  problem 

min  -  ^2  s(Xl)  such  that  /  e~s^dx  =  1,  s  €  Fu  n  Sv 

V  i=  1  ■' 

where  Su  =  e-splfl(7^)nlsc-fcns(J?n)  and  Fv  is  a  subset  of  the  lsc  functions  s  that  satisfies  f  e~s^dx  =  1, 
but  could  include  numerous  other  restrictions  exemplified  below  and  in  §5.  A  solution  s  of  this  problem 
provides  a  density  estimator  through  the  composition  e~s,  where  we  observe  that  the  nonnegativity  is 
automatically  satisfied.  We  refer  to  [29]  for  further  details  including  simplifications  in  the  approximate 
problem  that  ensure  its  convexity.  Two  numerical  examples  illustrate  the  approach. 


28 


0.15 


(a)  (b) 

Figure  9:  Normal  Example:  Sample  of  size  25  (a)  and  corresponding  exponential  epi-spline  (b). 

Normal  density  example.  Suppose  that  the  actual  density  h°  is  a  bivariate  normal  with  mean  vec¬ 
tor  (1,2)  and  variance-covariance  matrix  (1,  0.5;  0.5,  2);  see  Figure  8(a).  We  would  like  to  reproduce 
this  density  using  a  sample  of  size  v.  The  partition  1Z  consists  of  N  =  100  rectangles  of  equal  size 
covering  a  domain  defined  in  each  dimension  to  range  from  two  empirical  standard  deviations  below 
the  lowest  observed  value  to  two  empirical  standard  deviations  above  the  highest  value.  In  addition 
to  constraints  specified  below,  we  let  second-order  partial  derivatives  of  the  epi-splines  to  be  in  the 
range  [—1000,1000].  For  a  sample  size  of  v  =  10,000,  exponential  epi-spline  estimates  are  visually 
good  (not  displayed)  regardless  of  the  combination  of  additional  constraints  on  continuity,  continuous 
differentiability,  and  log-concavity  with  mean-square  errors  MSE  =  f(e~s lx'  —  h0(x))2h°(x)dx  of  ap¬ 
proximately  40.  A  reduced  sample  size  of  v  =  100,  provides  a  MSE  of  6.1  •  10°  and  a  poor  visual  fit 
under  a  continuity  constraint.  Under  continuous  differentiability,  the  MSE  improves  to  2.1  •  103  and 
a  good  visual  fit;  see  Figure  8(b).  However,  the  estimate  is  not  log-concave.  An  additional  constraint 
would  enforce  such  a  condition  easily,  but  instead  of  displaying  that  case  we  also  reduce  the  sample 
size.  Under  the  extremely  small  sample  with  v  =  25  illustrated  in  Figure  9(a),  a  satisfactory  fit  with 
MSE  of  5.3  •  103  is  obtained  under  continuous  differentiability  and  log-concavity  constraints  as  seen 
in  Figure  9(b).  Of  course,  the  normal  density  is  especially  well  suited  for  estimation  by  second-order 
exponential  epi-splines.  In  fact,  it  suffices  to  consider  a  partition  consisting  of  a  single  open  set,  Mn . 
We  next  consider  a  more  challenging  situation. 

Uniform  Mixture  Example.  Suppose  that  the  actual  density  hP(x)  =  4  if  x  =  ( x±,x2 )  satisfies 
(k  -  1)0.2  <  X!  <  (k  -  1)0.2  +  0.1,  k  =  1,2, ...,  5,  and  (l  -  1)0.2  <  x2  <  (l  -  1)0.2  +  0.1,  1  =  1,2, ...,  5, 
and  h°(x)  =  0  otherwise;  see  Figure  10(a)  for  a  color  contour  plot.  This  “uniform  mixture”  density  is 
estimated  by  an  exponential  epi-spline  defined  on  [0,  l]2.  Additional  information  about  the  support  of 
the  actual  density  is  ignored.  We  assume  that  the  partial  derivatives  of  the  epi-splines  on  the  open  sets 
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(b) 


Figure  10:  Uniform  Example:  True  density  (a)  and  exponential  epi-spline  with  N  =  2500  (b). 


of  the  partition  are  in  the  range  [—1, 1]  and  rely  on  a  sample  of  size  v  =  2500.  Figure  10(b)  shows  an 
exponential  epi-spline  estimate  for  a  partition  with  N  =  2500.  Although  the  MSE  is  large,  the  essential 
nature  of  the  density  is  captured.  The  fit  is  obtained  in  693  seconds  after  solving  a  problem  with  15,000 
variables,  a  convex  objective  function,  and  40,000  linear  inequality  constraints10.  Figure  11  provides 
similar  results,  obtained  in  30  seconds,  for  N  =  625  in  a  contour  plot  (a)  and  a  titled  view  (b). 


Figure  11:  Uniform  Example:  Exponential  epi-spline  with  N 


625  in  contour  (a)  and  tilted  view  (b). 


10We  refer  to  [29]  for  a  convex  formulation  that  removes  the  constraint  that  ensures  that  the  estimate  integrates  to  one 
and  replaces  it  by  a  penalty  term  in  the  objective  function. 
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Abstract.  The  definition  of  infinite  refinement  in  Royset  &;  Wets  (Multivariate  epi-splines  and  evolv¬ 
ing  function  identification  problems,  Set- Valued  and  Variational  Analysis,  2016)  needs  to  be  strengthen 
to  exclude  refinements,  as  illustrated  by  an  example,  that  would  not  become  fine  enough. 

Date:  April  6,  2016 


The  definition  of  an  infinite  refinement  needs  to  be  adjusted  as  follows: 

3.4  Definition  (infinite  refinement)  A  sequence  {'lZv}cJfi=l  of  partitions  of  a  closed  set  B  C  Mn,  with 
TZU  =  is  an  infinite  refinement  if 

for  every  x  £  B  and  £  >  0,  there  exist  v  £  IV  and  6  £  (0,  e)  such  that 
Rk  C  fB(x,  £ )  for  every  u  >  v  and  k  satisfying  Rk  n  lB(x,  5)  0. 

The  reason  is  that  the  corresponding  definition  in  [1]  allows  for  the  possibility  to  choose  a  somewhat 
pathological  refinement  that  would  fail  to  become  sufficiently  “small”  as  illustrated  next: 

Example.  Let  B  =  [0,  oo)  C  1R  and  the  infinite  refinement  be  partitioning  (0,  y/T)  mostly  into  segments 
of  length  1/s/v,  with  the  exception  that  a  portion  of  the  segment  nearest  the  origin  is  added  to  the 
final  segment  covering  (yC^oo).  Specifically,  we  let  Nv  =  u  +  1,  R\  =  (0, 1/(4^/^))  U  (l/(2y/zl),  l/yC-), 
Rvk  =  (( k  —  1 k/yR)  for  k  =  2, ...,  Nu  —  1,  and  RUNV  =  (yC-,  oo)  U  (1/(4^),  l/(2yd/)).  Clearly,  for 
every  u  £  IV,  {Rk}k=l  is  a  partition  of  B.  Also,  {TZu}^fi=1  would  be  an  appropriate  infinite  refinement 
of  B  according  to  [1],  which  requires  only: 

for  every  x  £  B  and  e  >  0,  there  exists  u  £  IV  such  that 
Rk  C  fB(x,£)  for  every  u  >  u  and  k  satisfying  x  £  cl Rvk. 

We  establish  that  this  requirement  is  satisfied:  Let  x  £  B  and  e  >  0  be  arbitrary.  First,  suppose  that 
x  =  0.  Then,  set  v  >  1/e2.  In  this  case,  we  have  that  x  £  cl R\  for  all  u.  Since  R\  C  [0, 1/yd/]  c  [0,e] 
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for  all  v  >  u,  the  requirement  R\  C  fB(x,e)  is  satisfied.  Second,  suppose  that  x  >  0.  Then,  set 
v  >  max{4/e2,  x2,  l/(4x’2),  1}.  In  this  case,  l/(2y/v)  <  x  <  y/u  for  v  >  v.  Consequently,  x  0  cl RVN„ 
for  any  v  >  v.  We  therefore  have  that  for  each  v  >  v,  there  exists  a  kv(x )  €  {1,2,  ...,NU  —  2}  such 
that  x  G  cl Rfcvrx\  U  cl Rkv(x)+i  anc^  x  dRk  for  all  k  G  {1,  2, ...,  Nu}  \  {ku(x),  ku(x)  +  1}.  Since  for 
each  v  >  v,  R^v^  U  is  contained  in  a  line  segment  of  length  at  most  e  the  requirement  holds. 

We  have  shown  that  satisfies  the  condition  of  the  original  definition  in  [1],  This  (pathologi¬ 

cal)  infinite  refinement  makes  the  proofs  of  Theorems  3.5,  3.8,  and  3.11  in  [1]  invalid  due  to  the  fact 
that  there  are  points  in  RvNV  that  are  arbitrarily  closed  to  0,  but  RvNV  is  not  becoming  arbitrarily 
“small.”  The  example  fails  the  condition  of  the  adjusted  definition  given  here  as  there  is  no  6  G  (0,  e) 
such  that  RvNv  n  IB (0,5)  =  0  for  all  v.  We  observe  that  all  “natural”  partitions  such  as  the  one  with 
R vk  =  ((k  —  l)/y/v,  k/y/v)  for  k  =  1  ,...,NV  —  1,  and  RVNU  =  (y/v,oo)  satisfies  the  condition  of  the 
adjusted  definition. 

With  the  adjusted  definition  of  infinite  refinement,  Theorems  3.5,  3.8  and  3.11  in  [1]  are  correct  as 
stated.  However,  we  provide  below  an  updated  version  of  the  proof  of  Theorem  3.5  to  pinpoint  the 
“sticky”  issue;  some  typos  are  also  corrected  and  we  provide  some  additional  explanation.  Corrected 
proofs  of  Theorems  3.8  and  3.11  follow  the  same  pattern. 

3.5  Theorem  (dense  approximation)  For  any  p  G  IV o  and  an  infinite  refinement  of  a  closed 

set  B  C  Fi"  , 

OO 

e-sp\^l(TZu)  is  dense  in  lsc-fcns(B). 

v=l 

Proof:  Let  s°  G  lsc-fcns(-B)  and  1ZV  =  {R’j(}^1.  It  suffices  to  construct  a  sequence  of  epi-splines  of 
order  p  =  0.  For  every  v  G  IV  and  Rvk ,  k  =  1,2, ...,  Nu,  we  define 

infeed Rvk  s°(x)  if  infx6ci^  s°(x)  G  [-v,  v\ 
aV(Rk)  '■=  <  v  if  infeed  rv  s°(x)  >  v, 

—v  otherwise 

and  construct  sv  :  Fln  — >  Fi  as  follows: 

su (x)  :=  ^min  ^au(Rk)  x  G  cl  Jft|,  iGU, 

and  su(x)  =  oo  for  x  0  B.  Clearly,  su  is  constant  on  each  Rvk,  k  =  1,2,  ...,NV  and  liminf,,./-^  su(x')  = 
su(x)  for  all  x  G  ]Rn .  Hence,  sv  G  e-spl({(7Cy)  and  consequently  also  in  e-spl{((7?.iy)  for  p  G  JV.  We  next 
show  that  the  two  conditions  of  Proposition  2.1  in  [1]  holds.  Let  iG8  and  e  >  0  be  arbitrary.  By  lower 
semicontinuity  of  s°  there  exists  5  >  0  such  that  s°(z)  >  s°(x)  —  e  whenever  z  G  fB (x,  5).  Since  {Ry}^=1 
is  an  infinite  refinement,  there  also  exist  v  and  7  G  (0,6)  such  that  Rk  C  fB(x,5)  for  every  v  >  V  and 
k  satisfying  Rk  n  F>(x ,7)  /  0.  Hence,  for  y  G  fB(x,  7/2)  n  B ,  su(y)  >  mhifc{min{i/,  infzeci^  s°(2:)}  : 
y  G  clift}  >  min{z/,  s°(x)  —  e}  for  all  u  >  u.  (It  is  the  existence  of  a  neighborhood  of  x  on  which 
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this  inequality  holds  that  might  fail  in  the  original  proof  resulting  in  the  possibility  that  the  following 
inequality  would  not  hold.)  Since  su(y)  =  oo  for  y  0  B,  we  thus  have  that  for  every  sequence  xv  — >  x, 

liminf^  su( xu)  >  lirninfj,  min{i/,  s°(x)  —  e}  =  s°(x)  —  e. 

Since  e  is  arbitrary,  liminf  su(xu)  >  s°(x).  Because  B  is  closed,  this  inequality  also  holds  for  x  0  B. 
Thus,  condition  a)  of  Proposition  2.1  in  [1]  is  established.  For  b),  simply  set  xv  =  x  for  all  v.  If  x  0  B, 
then  su(xu)  =  s°(x)  =  oo.  If  x  €  B,  then  su(x)  <  max{-!/,  s°(x)}.  From  this  follows  that  for  x  G  Mn, 

limsupy  sl'(x1')  =  limsupj,  su(x)  <  s°(x), 

which  establishes  part  b)  of  [1,  Proposition  2.1]  and  concludes  the  proof.  □ 
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